Logging experiments

Experiments are useful tools to A/B test different prompts and models for your LLM applications. This guide shows you how to log experiment results to Arize. We'll go through the following steps:

• Create a dataset
• Log an experiment

Setup dependencies

1. Install python dependencies
2. Import dependencies
3. Set up environment variables for your Arize Space ID, API Key, and Developer Key
4. Set up environment variables for your OpenAI API Key

Create dataset

We will be using a simple dataset with two columns: input and output.

Inputs are string values that you can pass to an LLM. Outputs are the expected responses that you can use later.

Log experiment

We will be logging an experiment with three columns:

• example_id is the dataset row ID, which is needed to map the results to the specific dataset row with inputs and expected outputs.
• result is the output of the LLM pipeline.
• correctness is the evaluation label of the experiment.