This notebook will walk through how you can use Portkey to seamlessly use different LLMs within the same application. It also adds Arize tracing/evals so you can observe and evaluate the different LLM calls.

Setup Arize Tracing with PortkeyInstrumentor

Setup application using Portkey Universal API for different LLM calls

This application runs a structured debate between two LLMs—one arguing “pro” and the other “con” on a given topic—while a third LLM acts as moderator to score each side and suggest prompt refinements. Over multiple iterations, the debate prompt is progressively improved to produce more balanced and persuasive arguments.

#Evals

Let's add some Arize Evals. Specifically we will add a toxicity eval to make sure the outputs from the debators aren't racist, sexist, chauvinistic, overly biased, or otherwise toxic.

Export Traces from Arize into Dataset

Create Evals

Export Evals dataset to Arize