Notebooks
A
Azure Azure Openai Getting Started

Azure Openai Getting Started

azure-openai-samplesBasic_Sampleslangchain
alph-notebooks/azure-openai-samples / azure_openai_getting_started.ipynb

Getting Started Azure OpenAI with LangChain in Python

The purpose of this notebook is to provide a step-by-step guide on how to set up with Azure OpenAI and use the available models through the LangChain framework in Python. Additionally, it will demonstrate, through examples, how to implement models and utilize available features.

Obtaining Keys and Endpoints

Using the Azure sign up page one can sign up and create an Azure OpenAI resource, giving access to all necessary credentials.

Setup

Installing and Importing Dependencies

Microsoft recommends using:

  • %pip for installing within Jupyter or IDE environments.
  • %conda for installing within Conda environments.

Install the most recent version of langchain and langchain_openai.

%pip install -U langchain langchain_openai

Install the most recent version of pandas and numpy.

%pip install -U pandas numpy

Install Packages in a Virtual Environment (Optional)

Set up a virtual environment by going to your project directory and executing the following command. This will create a new virtual environment in a folder named .venv.

MacOS/UNIX

  • python3 -m venv .venv

Windows

  • py -m venv .venv

Activating the Virtual Environment

To use the virtual environment, you must first activate it by executing the following command.

MacOS/UNIX

  • source .venv/bin/activate

Windows

  • .venv\Scripts\activate

Deactivating the Virtual Environment

If you want to leave the virtual environment in MacOS/UNIX and windows, simply execute the following command in the terminal:

deactivate

Import Packages

[13]

Set all required Environment Variables

Create a .env file and store your credentials like follows:

AZURE_OPENAI_API_KEY = <"Your key here">

AZURE_OPENAI_ENDPOINT = <"Your endpoint here">

Or use dotenv to set and load all required env variables into/from your .env file.

%pip install python-dotenv

[1]
(True, 'COMPLETIONS_MODEL', 'Your model here')

Get all required Environment Variables

[5]
True
[8]

Creating an AzureChatOpenAI Model

More information on LangChain's AzureChatOpenAI support can be found in the integration documentation.

  • Environment variable values can be passed as parameters.
  • Alternatively, if not passed in, the constructor will search for environment variables with corresponding names.
[9]

In the above code sample, OPENAI_API_VERSION and AZURE_OPENAI_ENDPOINT are both being passed in, but AZURE_OPENAI_API_KEY is being retrieved within the constructor.

Other Optional Parameters

  • temperature determines how creative and unpredictable, or how deterministic and predictable, the model should be in its responses. A temperature of 0 would be predictable, while anything higher would make responses more random.

  • max_tokens defines the maximum number of tokens (words or pieces of words) the model can generate in its response.

  • timeout specifies the maximum amount of time (in seconds) to wait for a response from the API before timing out. An APITimeoutError will be raised in the case of a timeout.

  • max_retries sets the number of times the API request should be retried in case of retriable failure before giving up.

  • model specifies the model to be used.

  • model_version indicates the specific version of the chosen model to use. This is useful for maintaining consistency in testing and for tracing purposes, such as tracking API calls or diagnosing issues related to specific model versions.

  • See the API Reference for more details.

  • Other parameters may be available in different SDK's.

Wait in between API calls

The number of API requests a user can make depends on their Azure plan and account settings. If too many requests are sent in a short period, an error may occur, prompting the user to wait for x amount of time before sending another request.

When creating a model, one of the key parameters is the max_retries setting. The underlying Python OpenAI library will automatically wait and retry the call on your behalf at least 2 times by default before raising a RateLimitError. This behavior can be adjusted by setting a different value for max_retries.

Visit the quotas and limits page to view detailed information related to account limits and restrictions.

Model Usage

Using Messages from the langchain_core.messages Library

The langchain_core.messages library allows the user to define messages for the model and assign roles to each message.

  • LangChain-compatible chat models take a list of messages as input and return the AI message as output.

  • All messages have role and content properties. In the sample below, the roles are set by using the SystemMessage and HumanMessage classes. We'll cover more on this later .

  • Additional provider-specific information can be incorporated using the additional_kwargs parameter. This could include provider-specific metadata or custom settings and flags.

[20]
[23]
[32]
'You have just created your first artificial intelligence model!'

Prompting

  • Prompts are the inputs to language models, refined from raw user inputs to be ready for processing by the models.

  • Prompting involves crafting text inputs that clearly communicate with the models, outlining the specific task we want it to accomplish. This can include:

    • Selecting the appropriate wording and setting a particular tone or style.
    • Providing necessary context.
    • Assigning a role, such as asking it to respond as if it were a native speaker of a certain language.

Prompt Templates

  • LangChain allows developers to design parameterized Prompt Templates that are reusable and easily transferable between different models for integration.

  • It takes user input and inserts said input into the prompt to feed into the language models.

PromptTemplate

PromptTemplate is used to create an instance of Prompt, and this is invoked by sending it to a model, which produces a PromptValue.

The example code uses .from_template, which handles a single string template with placeholders for dynamic inputs.

[15]
[16]
"In Rotorua, New Zealand, December falls in the Southern Hemisphere's summer, which is a great time for growing a variety of vegetables. Here are some vegetable crops you can plant in December:\n\n1. **Tomatoes**: Ideal for summer planting, they thrive in the warm weather.\n2. **Capsicums (Bell Peppers)**: These also enjoy the summer heat.\n3. **Zucchini**: Fast-growing and productive during warm months.\n4. **Cucumbers**: Perfect for summer salads and pickling.\n5. **Beans**: Both bush and pole beans grow well in the warm season.\n6. **Sweet Corn**: Requires warm temperatures and plenty of sunlight.\n7. **Pumpkins**: Plant now for a harvest in autumn.\n8. **Eggplants**: Another heat-loving crop.\n9. **Lettuce**: Opt for heat-tolerant varieties to avoid bolting.\n10. **Radishes**: Fast-growing and can"

ChatPromptTemplate

This is optimized for a conversation-like format. The prompt is a list of chat messages. Each chat message is associated with role and content. In the example code, .from_messages is used to include multiple messages.

Here, we will hardcode roles in the chat prompt, as opposed to using the pre-built roles SystemMessage or HumanMessage like earlier.

[16]
[18]
"Hi Lucy! New Zealand is a fantastic choice with its stunning landscapes, rich culture, and exciting activities. Here are some recommendations to make your trip memorable:\n\n### Natural Features\n1. **Fiordland National Park**: Home to the famous Milford Sound and Doubtful Sound, this area offers breathtaking fjords, waterfalls, and rainforests.\n2. **Tongariro National Park**: Known for its dramatic volcanic landscape, you can hike the Tongariro Alpine Crossing, one of the best one-day hikes in the world.\n3. **Rotorua**: Famous for its geothermal activity, you can see geysers, hot springs, and mud pools. Don't miss the Wai-O-Tapu Thermal Wonderland.\n4. **Aoraki/Mount Cook**: The highest mountain in New Zealand, offering stunning views, glaciers, and excellent hiking trails.\n5. **Bay of Islands**: A beautiful coastal area with over 140 subtropical islands, perfect for sailing, fishing,"

Assigning Roles Using LangChain Messages

Compared to hardcoding the roles like above, LangChain Messages allow for more flexibility and better management, especially with complex conversations involving multiple roles. It also simplifies the visualization of the conversation flow.

It is therefore recommended to use LangChain messages where possible.

Basic Message Types

SystemMessageSet how the AI should behave (appropriate wording, tone, style, etc.)
HumanMessageMessage sent from the user
AIMessageMessage from the AI chat model (context setting, guidance for response)

For more info, see Message Types and API Reference.

base message and MessagePromptTemplate

We can also pass a base message or MessagePromptTemplate instead of tuples.

[17]
[18]
'Yuzu is a popular fruit in Japan.'

MessagePlaceHolder

This is used to select which messages to include when formatting.

[50]

We can then input more prompts, which will take the MessagePlaceholders' place and create lines of sentences or a conversation.

[52]
[53]
'The next team meeting is on Tuesday at 11:00 AM. The attendees are Alice, Bob, Carol, David, and Emily.'

FewShotPrompt

We can use examples (shots) to condition the model for a better response by including some example input and output in the prompt. This will inform the model about the context and how we want the output to be formatted.

[55]
Currency Unit Conversion: [Input] one dollar => [Output] $1
Currency Unit Conversion: [Input] one hundred yen => [Output] ¥100

Chaining

  • Many LangChain components implement the Runnable protocol, which allows them to be easily chained together. These components can be combined in a sequence of calls, which we refer to as a chain.

  • Chaining Runnables in sequence.

[57]

This is how we have been using prompts, but now we will skip this step and invoke using the chain.

[59]
'In Japan, maguro (tuna) is the most popular sashimi. Globally, salmon sashimi tends to be more popular.'
[60]
In Japan, the most popular sashimi is often Maguro (tuna), specifically the fatty part known as Otoro. Globally, Salmon sashimi tends to be more popular due to its rich flavor and widespread availability.

Streaming Chat

Azure OpenAI chat models also support a Streaming Chat feature. This feature allows for text to be received sentence by sentence, rather than waiting for the entire response to arrive.

[7]
Certainly! The story of Papatuanuku and Ranginui is a central creation myth in Māori mythology, the indigenous belief system of the Māori people of New Zealand. It explains the origins of the world and the natural phenomena within it.

### The Story of Papatuanuku and Ranginui

In the beginning, there was nothing but darkness, a void known as Te Kore. From this void emerged two primordial beings: Ranginui, the Sky Father, and Papatuanuku, the Earth Mother. They lay tightly embraced, their union so close that no light could penetrate between them, and their many children were born in this eternal night.

Their children, who were gods of various natural elements and aspects of life, grew frustrated with the darkness. They longed for space and light. Among these children were Tāne Mahuta, the god of forests and birds; Tangaroa, the god of the sea; Tāwhirimātea, the god of storms and winds; and Tū

Check the costs and token usage of a given model API call

[52]

get_openai_callback() is a context manager of the OpenAICallbackHandler class, meaning it calls this class and creates an instance when used.

Below is an example of how to use the get_openai_callback(). However, to get an accurate estimation of cost, you must pass the model and model version as parameters to the AzureChatOpenAI constructor.

[53]
Total tokens used: 37
Total prompt tokens: 27
Total prompt tokens: 10
Total cost (in dollars): $0.000285
Total successful requests): 1

How to use structured outputs

Import the required packages. BaseModel is a parent class that all tools will inherit from, and Field is used to define all properties of the tool.

Tools

Tools are essentially classes that can be passed to a chosen model to influence or structure how the response should be formatted or generated.

For example:

  • A Weather tool with a specific API call could be passed so the model knows to use this specific API for data retrieval.
  • A City tool with fields like population, size, and main_language so the model can return any city-related queries with an object containing the corresponding filled fields.
  • An Image tool with a url field to be returned when asked to search for an image containing a dog, with the field containing the URL of the image.
[43]
[44]
Person(name='Kate Sheppard', alive=False, place_of_birth='Liverpool, England', noteable_features="Leader of the women's suffrage movement in New Zealand, instrumental in making New Zealand the first country to grant women the right to vote in 1893.", hobbies='Activism, writing, public speaking')
As the response of the invocation has been structured using the Person tool, the response can be accessed like a Person object.
[45]
Kate Sheppard
False
Liverpool, England
Leader of the women's suffrage movement in New Zealand, instrumental in making New Zealand the first country to grant women the right to vote in 1893.

JSON

Models can also be explicitly told to respond in a JSON structured format. This could then be used for future API calls or for easier access to information. However, the word "json" must be included in the message string.

[46]
'{\n    "name": "Jane Doe",\n    "alive": true,\n    "place_of_birth": "Springfield, Illinois, USA"\n}'

The response can then be formatted into a JSON object and accessed using normal JSON notation.

[47]
{'name': 'Jane Doe',
, 'alive': True,
, 'place_of_birth': 'Springfield, Illinois, USA'}

Image input

Models can be fed image files as their inputs.

When using data of different types, such as text in the SystemMessage and a file in the HumanMessage, it's necessary to specify a type header so the model knows how to interpret the data.

Additionally, the URL must be passed directly under the url content header, allowing the model to retrieve the image autonomously.

[11]
"The image depicts a stunning mountain landscape featuring a snow-capped peak reflected in a calm, clear lake. The mountains are rugged and majestic, with a mix of snow and rocky terrain. The lake in the foreground is serene, with the reflection of the mountains creating a mirror-like effect on the water's surface. The sky is clear with a few scattered clouds, adding to the overall tranquility and beauty of the scene. The area appears remote and untouched, emphasizing the natural beauty of the mountainous region."

Embeddings

Embeddings, particularly AzureOpenAIEmbeddings, are a natural language processing technique that converts text into mathematical or vector representations. These representations capture the semantic meaning of words, phrases, or entire texts. This transformation enables Azure OpenAI search services to utilize numerical similarities between texts, returning the most relevant search results for a given query.

Setup

Embeddings models use a different Azure resource, this model name will be set below as follows.

[18]

Create a model

[73]
[0.029870394617319107, -0.004167019855231047, 0.008153270930051804, -0.011176464147865772, -0.015433868393301964]

Searching

To leverage the power of embeddings we will transform the text into an easily accessable data structure known as a DataFrame from the pandas library.

[77]

Rename the columns and index to make the DataFrame clearer.

[79]
[81]
[83]
[85]

Here you can see that for each text it's corresponding embeddings have been set in the Embeddings column in the same row.

[89]
[-0.012673170305788517, -0.020036686211824417]

Now that we have the numerical representation, various Azure services can use this for searching. However, to demonstrate what happens 'under the hood,' we will implement a basic vector search manually.

Cosine similarity

Cosine similarity measures the similarity between two vectors by evaluating the cosine of the angle between them. In essence, it determines how close two vector points or lines are to each other. Vectors that are closer in space typically share a closer semantic meaning according to the model. This principle forms the core functionality of vector-based search using embeddings.

Setup

The numpy library introduces some conveniant mathematical functions.

[100]

Search

Use embeddings to get the vector representation of a query.

[105]
[111]
[119]

Et voilà! The text with the highest cosine similarity discusses space and space exploration, which closely aligns with our query. We also observe that the second-most similar text mentions the Sun. While it’s not an exact match to our query, the Sun is a star, making it contextually similar and resulting in a relatively high cosine similarity.

Next Steps/Additional resources

Azure OpenAI Service

  • LangChain Azure OpenAI Docs
  • LangChain AzureChatOpenAI Docs
  • LangChain AzureChatOpenAI API
  • LangChain AzureOpenAIEmbeddings Docs
  • LangChain AzureOpenAIEmbeddings API

Azure AI Search

  • LangChain Azure AI Search Docs
  • LangChain Azure AI Search API

Azure Cosmos DB

  • LangChain AzureCosmosDBMongovCore Docs
  • LangChain AzureCosmosDBMongovCore API
  • LangChain AzureCosmosDBNoSQL Docs
  • LangChain AzureCosmosDBNoSQL API

Azure AI Document Intelligence

  • LangChain AzureAIDocumentIntelligenceLoader Docs
  • LangChain AzureAIDocumentIntelligenceLoader API

Azure AI Services/Azure AI Vision/Azure AI Speech

  • LangChain AzureAIServicesToolkit Docs
  • LangChain AzureAIServicesToolkit API

References

  • Azure Open AI Embeddings Section source

  • AzureChatOpenAI Structured Output source