Notebooks
L
Langfuse Integration Mistral Sdk

Integration Mistral Sdk

observabilityllmsgenaicookbookprompt-managementhacktoberfestlarge-language-modelsnextraLangfuselangfuse-docs
alph-notebooks/langfuse-docs / integration_mistral_sdk.ipynb

Cookbook: Mistral SDK Integration (Python)

This cookbook provides step-by-step examples of integrating Langfuse with the Mistral AI SDK (v1) in Python. By following these examples, you'll learn how to seamlessly log and trace interactions with Mistral's language models, enhancing the transparency, debuggability, and performance monitoring of your AI-driven applications.

Note: Langfuse is also natively integrated with LangChain, LlamaIndex, LiteLLM, and other frameworks. If you use one of them, any use of Mistral models is instrumented right away.

Overview

In this notebook, we will explore various use cases where Langfuse can be integrated with Mistral AI SDK, including:

  • Basic LLM Calls: Learn how to wrap standard Mistral model interactions with Langfuse's @observe decorator for comprehensive logging.
  • Chained Function Calls: See how to manage and observe complex workflows where multiple model interactions are linked together to produce a final result.
  • Async and Streaming Support: Discover how to use Langfuse with asynchronous and streaming responses from Mistral models, ensuring that real-time and concurrent interactions are fully traceable.
  • Function Calling: Understand how to implement and observe external tool integrations with Mistral, allowing the model to interact with custom functions and APIs.

For more detailed guidance on the Mistral SDK or the @observe decorator from Langfuse, please refer to the Mistral SDK repo and the Langfuse Documentation.

Setup

[ ]
[1]
[3]

Examples

Completions

We are integrating the Mistral AI SDK with Langfuse using the @observe decorator, which is crucial for logging and tracing interactions with large language models (LLMs). The @observe(as_type="generation") decorator specifically logs LLM interactions, capturing inputs, outputs, and model parameters. The resulting mistral_completion method can then be used across your project.

[6]

Optionally, other functions (api handlers, retrieval functions, ...) can be also decorated.

Simple Example

In the following example, we also added the decorator to the top-level function find_best_painter_from. This function calls the mistral_completion function, which is decorated with @observe(as_type="generation"). This hierarchical setup hels to trace more complex applications which involve multiple LLM calls and other non-llm methods which are decorated with @observe.

You can use langfuse.update_current_generation or langfuse.update_current_trace to add additional details such as input, output, and model parameters to the trace.

[ ]

Chained Completions

This example demonstrates chaining multiple LLM calls using the @observe decorator. The first call identifies the best painter from a specified country, and the second call uses that painter's name to find their most famous painting. Both interactions are logged by Langfuse as we use the wrapped mistral_completion method created above, ensuring full traceability across the chained requests.

[ ]

Streaming Completions

The following example demonstrates how to handle streaming responses from the Mistral model using the @observe(as_type="generation") decorator. The process is similar to the Completion example but includes handling streamed data in real-time.

Just like in the previous example, we wrap the streaming function with the @observe decorator to capture the input, model parameters, and usage details. Additionally, the function processes the streamed output incrementally, updating the Langfuse context as each chunk is received.

[ ]

Async Completion

This example showcases the use of the @observe decorator in an asynchronous context. It wraps an async function that interacts with the Mistral model, ensuring that both the request and the response are logged by Langfuse. The async function allows for non-blocking LLM calls, making it suitable for applications that require concurrency while maintaining full observability of the interactions.

[ ]

Async Streaming

This example demonstrates the use of the @observe decorator in an asynchronous streaming context. It wraps an async function that streams responses from the Mistral model, logging each chunk of data in real-time.

[ ]

Tool Calling

This snippet introduces Mistral's function-calling capability, where you can define custom functions to retrieve specific data, like payment status and date, based on a transaction ID. These functions are then registered with the Mistral model, allowing it to call them when processing queries. For a deeper dive into function calling with Mistral, refer to the official Mistral documentation.

[12]

The check_transaction_status function demonstrates the use of Mistral's function-calling capabilities. The function's result is then incorporated into the LLM's response, which is logged and traced in Langfuse. This example illustrates how external function calls can be seamlessly integrated into a Langfuse by using the wrapped mistral_completion function, ensuring that every step — from tool selection to final output - is captured for thorough observability.

[ ]