Pinecone 04 Langchain Chat

04 Langchain Chat

vector-databasesemantic-searchlearnAILLMgenerationPythonhandbookjupyter-notebookpinecone-exampleslangchain

alph-notebooks/pinecone-examples / 04-langchain-chat.ipynb

Export

Run Notebooks

Contents

No cells yet

Add cells to see them here

[ ]

   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 16.0 MB/s eta 0:00:00
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.5/2.5 MB 38.5 MB/s eta 0:00:00
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.3/65.3 kB 2.0 MB/s eta 0:00:00
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 363.0/363.0 kB 14.1 MB/s eta 0:00:00
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.2/45.2 kB 2.3 MB/s eta 0:00:00
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.9/50.9 kB 2.5 MB/s eta 0:00:00

We'll start by initializing the ChatOpenAI object. For this we'll need an OpenAI API key. Note that there is naturally a small cost to running this notebook due to the paid nature of OpenAI's API access.

[2]

Enter your OpenAI API key: ··········

Initialize the ChatOpenAI object. We'll set temperature=0 to minimize randomness and make outputs repeatable.

[3]

Chats with the Chat-GPT model gpt-4.1-mini are typically structured like so:

System: You are a helpful assistant.

User: Hi AI, how are you today?

Assistant: I'm great thank you. How can I help you?

User: I'd like to understand string theory.

The final "Assistant:" without a response is what would prompt the model to continue the conversation. In the official OpenAI ChatCompletion endpoint these would be passed to the model in a format like:

[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hi AI, how are you today?"},
    {"role": "assistant", "content": "I'm great thank you. How can I help you?"},
    {"role": "user", "content": "I'd like to understand string theory."}
]

In LangChain there is a slightly different format. We use three message objects like so:

[4]

Hello! I'm just a bunch of code, so I don't have feelings, but I'm here and ready to help you. How can I assist you today?

The format is very similar, we're just swapping the role of "user" for HumanMessage, and the role of "assistant" for AIMessage.

We generate the next response from the AI by passing these messages to the ChatOpenAI object.

[ ]

Certainly! String theory is a theoretical framework in physics that attempts to reconcile quantum mechanics and general relativity, aiming to provide a unified description of all fundamental forces and particles.

Here’s a basic overview:

1. **Fundamental Idea**: Instead of viewing the smallest building blocks of the universe as point-like particles (like electrons or quarks), string theory proposes that these are tiny, one-dimensional "strings" that can vibrate at different frequencies. Each vibration mode corresponds to a different particle.

2. **Dimensions**: While we experience the universe in 3 spatial dimensions plus time, string theory requires additional spatial dimensions—typically 10 or 11 total dimensions—to be mathematically consistent. These extra dimensions are thought to be compactified or curled up at scales too small to detect.

3. **Types of Strings**: Strings can be open (with two endpoints) or closed (forming loops). Different types of strings and their vibrations give rise to different particles, including force carriers like photons and gravitons.

4. **Unification**: One of the main goals of string theory is to unify all fundamental forces—gravity, electromagnetism, the strong nuclear force, and the weak nuclear force—into a single framework.

5. **Supersymmetry**: Many versions of string theory incorporate supersymmetry, a proposed symmetry between bosons (force-carrying particles) and fermions (matter particles). This helps solve certain theoretical problems and predicts partner particles for those we know.

6. **Current Status**: String theory is still a work in progress. It has not yet been experimentally confirmed, partly because the energy scales involved are far beyond current experimental capabilities. However, it remains a leading candidate for a theory of quantum gravity.

If you want, I can explain any of these points in more detail or discuss related topics!

Because res is just another AIMessage object, we can append it to messages, add another HumanMessage, and generate the next response in the conversation.

[7]

'Great question! Physicists believe string theory has the potential to produce a "unified theory"—often called a "Theory of Everything"—because of several key reasons:\n\n1. **All Particles as Vibrations of Strings**: In string theory, every fundamental particle is just a different vibrational mode of the same basic object: a string. This means that matter particles (like electrons and quarks) and force-carrying particles (like photons and gravitons) are all manifestations of the same underlying entity. This naturally unifies the description of particles and forces.\n\n2. **Inclusion of Gravity**: Unlike the Standard Model of particle physics, which successfully describes three of the four fundamental forces but does not include gravity, string theory inherently includes a particle that behaves like the graviton—the hypothetical quantum particle that mediates gravity. This is a major step toward unifying gravity with quantum mechanics.\n\n3. **Mathematical Consistency**: String theory requires a consistent mathematical framework that only works if all forces and particles fit together in a specific way. This constraint means that the theory naturally combines the different forces into a single framework, rather than treating them separately.\n\n4. **Supersymmetry and Extra Dimensions**: The incorporation of supersymmetry and extra spatial dimensions allows string theory to resolve many theoretical problems that arise when trying to combine quantum mechanics and gravity. These features help unify the forces by embedding them in a higher-dimensional space where they appear as different aspects of the same fundamental structure.\n\n5. **No Free Parameters**: Unlike some other theories, string theory is highly constrained. The properties of particles and forces emerge from the geometry and topology of the extra dimensions and the way strings vibrate, rather than being put in by hand. This suggests a deep underlying unity.\n\nIn summary, physicists see string theory as a promising candidate for a unified theory because it provides a single, coherent framework that naturally includes all known particles and forces—including gravity—arising from one fundamental entity: the string. However, it’s important to note that this is still a theoretical proposal and has yet to be experimentally verified.\n\nWould you like me to explain any of these points further?'

New Prompt Templates

Alongside what we've seen so far there are also three new prompt templates that we can use. Those are the SystemMessagePromptTemplate, AIMessagePromptTemplate, and HumanMessagePromptTemplate.

These are simply an extension of Langchain's prompt templates that modify the returning "prompt" to be a SystemMessage, AIMessage, or HumanMessage object respectively.

For now, there are not a huge number of use-cases for these objects. However, they can be useful if:

You want different types of response; AND
The types of response should depend on a set of pre-determined input values; AND
You want to save tokens by not explicitly specifying every possible type of input value in the prompts.

This will make more sense with an example. Suppose you want to tailor responses to people from a wide variety of countries. E.g. an LLM powered worldwide translator!

Some of the languages listed have been commented out as this is just an illustrative example, but the idea is that we can have many languages and dynamically alter the HumanMessage prompt so that we don't have to list all of them every time.

[8]

First let's see what the prompt looks like with single example.

[9]

ChatPromptValue(messages=[HumanMessage(content='Translate this input <INPUT_START> I hope when you come the weather will be clement. <INPUT_END>  into Esperanto. Do not include any other text in your response.', additional_kwargs={}, response_metadata={})])

Note that to use HumanMessagePromptTemplate as typical a prompt templates with the .format_prompt method, we needed to pass it through a ChatPromptTemplate object. This is case for all of the new chat-based prompt templates.

Using this we return a ChatPromptValue object. This can be formatted into a list or string like so:

[10]

[HumanMessage(content='Translate this input <INPUT_START> I hope when you come the weather will be clement. <INPUT_END>  into Esperanto. Do not include any other text in your response.', additional_kwargs={}, response_metadata={})]

[11]

'Human: Translate this input <INPUT_START> I hope when you come the weather will be clement. <INPUT_END>  into Esperanto. Do not include any other text in your response.'

Okay, let's see this new approach in action with our list of languages.

[12]


=== Response in English ===
I hope when you come the weather will be mild.
==================================================

=== Response in Esperanto ===
Mi esperas, ke kiam vi venos, la vetero estos milda.
==================================================

=== Response in Spanish ===
Espero que cuando vengas el tiempo sea benigno.
==================================================

Excellent!

As you can see, it's successfully translated into different languages based on our inputs, and we didn't have to use unnecessary tokens by inserting the entire language list into the prompt.

What if the outputs we need are more complicated? For example, what if the input information is technical information that needs to be formatted in a very specific way for the output?

E.g. Say that we want to:

Input technical information.
Only translate part of the technical information, not all of the text.
Maintain the same input structure in the output structure.

We can use the prompt templates approach for building an initial system message with a few examples for the chatbot to follow — few-shot training via examples. Let's see what that looks like.

[13]


=== Technical Translation in English ===
Status: 500 Internal Server Error
Response: {
    'error': 'Database connection failed',
    'code': 'DB_001',
    'timestamp': '2024-03-20T10:30:00Z'
}

Technical Note: This error occurs when the application cannot connect to the database.
================================================================================

=== Technical Translation in Esperanto ===
Status: 500 Internal Server Error
Response: {
    'error': 'Database connection failed',
    'code': 'DB_001',
    'timestamp': '2024-03-20T10:30:00Z'
}

Technical Note: Ĉi tiu eraro okazas kiam la aplikaĵo ne povas konekti al la datumbazo.
================================================================================

=== Technical Translation in Spanish ===
Status: 500 Internal Server Error
Response: {
    'error': 'Database connection failed',
    'code': 'DB_001',
    'timestamp': '2024-03-20T10:30:00Z'
}

Nota técnica: Este error ocurre cuando la aplicación no puede conectarse a la base de datos.
================================================================================

Perfect, we seem to get a good response!

Now, it's arguable as to whether all of the above is better than simple f-strings like:

[14]


=== Technical Translation in English ===
Status: 500 Internal Server Error
Response: {
    'error': 'Database connection failed',
    'code': 'DB_001',
    'timestamp': '2024-03-20T10:30:00Z'
}

Technical Note: This error occurs when the application cannot connect to the database.
================================================================================

=== Technical Translation in Esperanto ===
Status: 500 Internal Server Error
Response: {
    'error': 'Database connection failed',
    'code': 'DB_001',
    'timestamp': '2024-03-20T10:30:00Z'
}

Technical Note: Ĉi tiu eraro okazas kiam la aplikaĵo ne povas konekti al la datumbazo.
================================================================================

=== Technical Translation in Spanish ===
Status: 500 Internal Server Error
Response: {
    'error': 'Database connection failed',
    'code': 'DB_001',
    'timestamp': '2024-03-20T10:30:00Z'
}

Nota técnica: Este error ocurre cuando la aplicación no puede conectarse a la base de datos.
================================================================================

In this example, the above is far simpler. So we wouldn't necessarily recommend using prompt templates over f-strings in all scenarios.

One example where Prompt Templates might prove useful is in interpreting specific template format types. For example, suppose a project uses lots of jinja templates. Rather than writing our functions that handle the input values, f-strings and which renders the jinja template, LangChain Prompt Templates do all of this for us:

[15]


=== Technical Translation in English ===
Status: 500 Internal Server Error
Response: {{
    'error': 'Database connection failed',
    'code': 'DB_001',
    'timestamp': '2024-03-20T10:30:00Z'
}}

Technical Note: This error occurs when the application cannot connect to the database.
================================================================================

=== Technical Translation in Esperanto ===
Status: 500 Internal Server Error
Response: {{
    'error': 'Database connection malsukcesis',
    'code': 'DB_001',
    'timestamp': '2024-03-20T10:30:00Z'
}}

Teknika Noto: Ĉi tiu eraro okazas kiam la aplikaĵo ne povas konekti al la datumbazo.
================================================================================

=== Technical Translation in Spanish ===
Status: 500 Internal Server Error
Response: {{
    'error': 'Fallo en la conexión a la Database',
    'code': 'DB_001',
    'timestamp': '2024-03-20T10:30:00Z'
}}

Nota Técnica: Este error ocurre cuando la aplicación no puede conectarse a la Database.
================================================================================

Let's see what the prompts look like after LangChain interprets the Jinja2 templates. This demonstrates how LangChain automatically handles the template interpretation for us:

[16]


=== Formatted Prompt for Spanish ===

SYSTEM MESSAGE:
----------------------------------------
You are a technical translator. You must maintain the exact same format and structure in your translations.
    Only translate the explanatory text, keeping all technical terms, numbers, and formatting unchanged.
    
    Example input and output pairs:
    
    Input: "Error 404: Page not found"
    Output: "Error 404: Página no encontrada"
    
    Input: "Status: 200 OK
    Response: {{
        'data': 'success',
        'message': 'Operation completed'
    }}"
    Output: "Status: 200 OK
    Response: {{
        'data': 'success',
        'message': 'Operación completada'
    }}"
    
================================================================================

HUMAN MESSAGE:
----------------------------------------
Translate this technical information to SPANISH:
    
    Status: 500 Internal Server Error
    Response: {{
        'error': 'Database connection failed',
        'code': 'DB_001',
        'timestamp': '2024-03-20T10:30:00Z'
    }}
    
    Technical Note: This error occurs when the application cannot connect to the database.
    
    
    Note: Please use formal Spanish for technical documentation.
    
    
    
    Keep the term "DB_001" unchanged in the translation.
    
    Keep the term "Internal Server Error" unchanged in the translation.
    
    Keep the term "Database connection" unchanged in the translation.
    
    
================================================================================

Let's break down how LangChain automatically interpreted the Jinja2 templates in our prompts:

Language Filter and Variable:
- Original: {{ language|upper }}
- Interpreted as: SPANISH
- The |upper filter automatically converted the language to uppercase

Conditional Logic:

Original:

{% if language == 'spanish' %}
Note: Please use formal Spanish for technical documentation.
{% elif language == 'french' %}
Note: Please use formal French for technical documentation.
{% else %}
Note: Please maintain a formal tone in the translation.
{% endif %}

Interpreted as: Note: Please use formal Spanish for technical documentation.
The if statement automatically selected the Spanish-specific note

Loop Structure:

Original:

{% for term in technical_terms %}
Keep the term "{{ term }}" unchanged in the translation.
{% endfor %}

Interpreted as three separate lines, one for each technical term:

Keep the term "DB_001" unchanged in the translation.
Keep the term "Internal Server Error" unchanged in the translation.
Keep the term "Database connection" unchanged in the translation.

The for loop automatically iterated through our list of technical terms

Raw JSON Blocks:
- Original: {% raw %}{{ ... }}{% endraw %}
- Interpreted as: {{ ... }}
- The raw tags were automatically removed while preserving the JSON structure