Tracing via OTLP using Arize

This guide demonstrates how to use Arize for monitoring and debugging your LLM using Traces and Spans. We're going to build a simple question-and-answer application using LangChain and retrieval-augmented generation (RAG) to answer questions about the Arize documentation. Arize makes your LLM applications observable by visualizing the underlying structure of each call to your query engine and surfacing problematic spans of execution based on latency, token count, or other evaluation metrics. You can read more about LLM tracing here.

In this tutorial, you will:

1. Use opentelemetry and openinference to instrument our application in order to send traces via OTLP to Arize.
2. Build a simple question-and-answer application using LangChain and RAG to answer questions about the Arize documentation
3. Inspect the traces and spans of your application to identify sources of latency and cost

ℹ️ This notebook requires:

• An OpenAI API key
• An Arize Space & API Key (explained below)

Step 1: Install Dependencies 📚

Let's get the notebook setup with dependencies.

Step 2: OTLP Instrumentation

Step 2.a: Define an exporter to Arize

Creating an Arize exporter is very simple. We just need 2 things:

• Space and API keys, that will be sent as headers
• Model ID and version, sent as resource attributes

Copy the Arize API_KEY and SPACE_ID from your Space Settings page (shown below) to the variables in the cell below. We will also be setting up some metadata to use across all logging.

Next, we create an OTLP exporter with the Arize endpoint detailed above. Note that we use GRPC to export traces to Arize, which acts as a collector.

Step 2.b: Define a trace provider and initiate the instrumentation

Step 3: Build Your LangChain RAG Application 📁

Let's import the dependencies we need

Set your OpenAI API key if it is not already set as an environment variable.

This example uses a RetrievalQA chain over a pre-built index of the Arize documentation, but you can use whatever LangChain application you like.

Download the pre-built index from cloud storage and instantiate your storage context.

Let's test our app by asking a question about the Arize documentation:

Great! Our application works!

Step 4: Use our instrumented chain

We will download a dataset of queries for our RAG application to answer and see the traces appear in Arize.

Step 5: Log into Arize and explore your application traces 🚀

Log into your Arize account, and look for the model with the same model_id. You are likely to see the following page if you are sending a brand new model. Arize is processing your data and your model will be accessible for you to explore your traces in no time.

After the timer is completed, you are ready to navigate and explore your traces