Notebooks
L
Langfuse Integration Openai Sdk

Integration Openai Sdk

observabilityllmsgenaicookbookprompt-managementhacktoberfestlarge-language-modelsnextraLangfuselangfuse-docs
alph-notebooks/langfuse-docs / integration_openai_sdk.ipynb

Cookbook: OpenAI Integration (Python)

Python JS/TS

This is a cookbook with examples of the Langfuse Integration for OpenAI (Python).

Follow the integration guide to add this integration to your OpenAI project.

Setup

The integration is compatible with OpenAI SDK versions >=0.27.8. It supports async functions and streaming for OpenAI SDK versions >=1.0.0.

[ ]
[ ]
[2]

Examples

Chat completion (text)

[3]

Chat completion (image)

Simple example using the OpenAI vision's functionality. Images may be passed in the user messages.

[4]

Go to https://cloud.langfuse.com or your own instance to see your generation.

Chat completion

Link to trace in Langfuse

Chat completion (streaming)

Simple example using the OpenAI streaming functionality.

[5]
Why don't scientists trust atoms?

Because they make up everything!None

Chat completion (async)

Simple example using the OpenAI async client. It takes the Langfuse configurations either from the environment variables or from the attributes on the openai module.

[5]
[6]

Go to https://cloud.langfuse.com or your own instance to see your generation.

Chat completion

Link to trace in Langfuse

Functions

Simple example using Pydantic to generate the function schema.

[ ]
[9]
[13]

Go to https://cloud.langfuse.com or your own instance to see your generation.

Function

Link to trace in Langfuse

Langfuse Features (User, Tags, Metadata, Session)

You can access additional Langfuse features by adding the relevant attributes to the OpenAI request. The Langfuse integration will parse these attributes. See docs for details on all available features.

[15]

AzureOpenAI

The integration also works with the AzureOpenAI and AsyncAzureOpenAI classes.

[ ]
[ ]
[ ]
[ ]

Group multiple generations into a single trace

Many applications require more than one OpenAI call. The @observe() decorator allows you to nest all LLM calls of a single API invocation into the same trace in Langfuse.

[ ]

Fully featured: Interoperability with Langfuse SDK

The trace is a core object in Langfuse and you can add rich metadata to it. See Python SDK docs for full documentation on this.

Some of the functionality enabled by custom traces:

  • custom name to identify a specific trace-type
  • user-level tracking
  • experiment tracking via versions and releases
  • custom metadata
[ ]

Programmatically add scores

You can add scores to the trace, to e.g. record user feedback or some programmatic evaluation. Scores are used throughout Langfuse to filter traces and on the dashboard. See the docs on scores for more details.

The score is associated to the trace using the trace_id.

[14]