Notebooks
L
Langfuse Integration Amazon Bedrock

Integration Amazon Bedrock

observabilityllmsgenaicookbookprompt-managementhacktoberfestlarge-language-modelsnextraLangfuselangfuse-docs
alph-notebooks/langfuse-docs / integration_amazon_bedrock.ipynb

sidebarTitle: Amazon Bedrock seoTitle: Open Source Observability and Metrics for Amazon Bedrock description: Open source observability for Amazon Bedrock applications and the Bedrock SDK. logo: /images/integrations/bedrock_icon.png

Amazon Bedrock Integration

Amazon Bedrock is a fully managed AWS service that lets you use foundation models and custom models to generate text, images, and audio.

When using Langfuse with Amazon Bedrock, you can easily capture detailed traces and metrics for every request, giving you insights into the performance and behavior of your application.

All in-ui Langfuse features next to tracing (playground, llm-as-a-judge evaluation, prompt experiments) are fully compatible with Amazon Bedrock – just add your Bedrock configuration in the project settings.

Integration Options

There are a few ways through which you can capture traces and metrics for Amazon Bedrock:

  1. via an application framework that is integrated with Langfuse:

  2. via a Proxy such as LiteLLM

  3. via wrapping the Bedrock SDK with the Langfuse Decorator (see example below)

How to wrap Amazon Bedrock SDK (Converse API)

[96]

Authenticate AWS Session

Sign in with your AWS Role that has access to Amazon Bedrock.

[103]
[104]
EU Anthropic Claude 3 Sonnet - eu.anthropic.claude-3-sonnet-20240229-v1:0
EU Anthropic Claude 3 Haiku - eu.anthropic.claude-3-haiku-20240307-v1:0
EU Anthropic Claude 3.5 Sonnet - eu.anthropic.claude-3-5-sonnet-20240620-v1:0
EU Meta Llama 3.2 3B Instruct - eu.meta.llama3-2-3b-instruct-v1:0
EU Meta Llama 3.2 1B Instruct - eu.meta.llama3-2-1b-instruct-v1:0

Set Langfuse Credentials

[44]

With the environment variables set, we can now initialize the Langfuse client. get_client() initializes the Langfuse client using the credentials provided in the environment variables.

[ ]

Wrap Bedrock SDK

[88]

Run Example

[105]
Anthropic
Understood. I'll continue to act as Alex, the AI personal finance advisor from SmartFinance Advisors, maintaining that character throughout our interaction. I'll provide financial advice and guidance based on the user's questions and needs. If I'm unsure about something, I'll ask for clarification as instructed. How may I assist you with your financial matters today?

Llama3-2
Hello again! I'm glad you're excited about receiving my advice. How can I assist you with your financial goals today? Are you looking to create a budget, paying off debt, saving for a specific goal, or something else entirely?

Can I monitor Amazon Bedrock cost and token usage in Langfuse?

Yes, you can monitor cost and token usage of your Bedrock calls in Langfuse. The native integrations with LLM application frameworks and the LiteLLM proxy will automatically report token usage to Langfuse.

If you use the Langfuse Decorator or Context Manager, you can report token usage and (optionally) also cost information directly. See example above for details.

You can define custom price information via the Langfuse dashboard or UI (see docs) to adjust to the exact pricing of your models on Amazon Bedrock.

Additional Resources