Bedrock Tracing

This guide demonstrates how to use Arize for monitoring and debugging your LLM using Traces and Spans. You can read more about LLM tracing here. In this tutorial, you will use opentelemetry and openinference to instrument our application in order to send traces to Arize.

ℹ️ This notebook requires:

• An AWS account
• An Arize Space & API Key

Step 1: Install Dependencies 📚

Let's get the notebook setup with dependencies.

Step 2: Get your API Keys

Copy the Arize API_KEY and SPACE_ID from your Space Settings page (shown below) to the variables in the cell below.

Follow this guide for setting up your AWS credentials. You will need to enable model access for Bedrock here.

Step 3. Add our tracing code

We will be using the arize-otel package to register the URL and authentication parameters to send to Arize using OpenTelemetry. You can see what's under the hood by looking here.

Step 4: Run your LLM application

Let's test our app by asking to write a haiku. If you have difficulty invoking the model, you can change the modelId to the ARN of a model that you have access to (see guide here).

The invocation parameters are also different for Bedrock vs. traditional LLM inference. Here's the Bedrock docs for invoke_model vs. converse.

This also works with invoke_model.

Here's an example with boto3 agents. You will need to set up an agent in the AWS console, make sure it's deployed, and then create an alias for it. Make sure the region_name is correct or else the agent will not work.

Example URL to access the console: https://us-east-2.console.aws.amazon.com/bedrock/home?region=us-east-2#agents

Step 5: Log into Arize and explore your application traces 🚀

Log into your Arize account, and look for the project with the same project_name.