Getting Started with mistral-inference

This notebook will guide you through the process of running Mistral models locally. We will cover the following:

• How to chat with Mistral 7B Instruct
• How to run Mistral 7B Instruct with function calling capabilities

We recommend using a GPU such as the A100 to run this notebook.

Download Mistral 7B Instruct

Chat with the model

Function calling

Mistral 7B Instruct v3 also supports function calling!

Let's start by creating a function calling example

Since we have already loaded the tokenizer and the model in the example above. We will skip these steps here.

Now we can encode the message with our tokenizer using MistralTokenizer.

and run generate to get a response. Don't forget to pass the EOS id!

Finally, we can decode the generated tokens.