Notebooks
G
Groq
Parallel Tool Use

Parallel Tool Use

managed-by-terraformtutorials02-tool-useparallel-tool-usegroq-api-cookbook

Parallel Tool Use with Groq API

What are Tools and Tool Use? 🤔

To extend the capabilities of Large Language Models (LLMs) in AI-powered applications and systems, we can provide tools to allow them to interact with external resources (e.g. APIs, databases, web) by:

  • Providing tools (or predefined functions) to our LLM
  • Defining how the tools we provide should be used to teach our LLM how to use them effectively (e.g. defining input and output formats)
  • Letting the LLM autonomously decide whether or not the provided tools are needed for a user query by evaluating the user query, determining whether the tools can enhance its response, and utilizing the tools accordingly

By providing our LLMs with tools, we can enable them with the option to gather dynamic data that they wouldn't otherwise have access to in their pre-trained, or static, state.

What is Parallel Tool Use? 🧰

Let's take tool use a step further. Imagine a workflow where multiple tools can be used simultaneously, enabling more efficient and effective responses. This concept, known as parallel tool use, is key for building agentic workflows that can deal with complex queries and tasks.

By leveraging parallelism, you can build applications that can call multiple tools concurrently and then combine the outputs of these tools in parallel for your application to provide an accurate response or complete a task for a complex query.

Parallel Tool Use and Function Calling with Groq API 💪

Tool use, or function calling, support is available for all text models and parallel tool use support is enabled for all Llama 3 and Llama 3.1 models. The Llama 3.1 models now support the native tool use format that was used in post-training, which results in much better quality, especially in multi-turn conversations and parallel tool calling.

For your applications that require tool use, we highly recommend trying the following models:

  • meta-llama/llama-4-scout-17b-16e-instruct
  • meta-llama/llama-4-maverick-17b-128e-instruct
  • llama-3.3-70b-versatile
  • llama-3.1-70b-versatile
  • llama-3.1-8b-instant

In this basic tutorial, we will be reviewing the basic structure of parallel tool use with Groq API to fetch given product pricing for multiple bakery items.

Tutorial Setup

Make sure you have ipykernel and pip installed before running the following pip command to install required dependencies:

[ ]

Define Tools

Next, create a .env file in the root directory of this project and populate it with your GROQ_API_KEY. If you don't have one, you can create an account on GroqCloud and generate one for free at https://console.groq.com. Your .env file should have a line that looks like the following:

GROQ_API_KEY=gsk_...

We will use Python-dotenv to read our Groq API key from the .env file. It is a best practice to keep your API key stored and secure in a .env file instead of your main application file in case you make code public, which could cause you to leak your API key and lead to API key abuse!

[ ]

For this tutorial, we will use the llama-3.3-70b-versatile model:

[ ]

Let's define a tool, or function, that the LLM can invoke to fetch pricing for bakery items:

[ ]

Now let's define our system messages and tools before running the chat completion:

[ ]

We've set the tool_choice parameter to auto to allow our model to choose between generating a text response or using the given tools, or functions, to provide a response. This is the default when tools are available.

We could also set tool_choice to none so our model does not invoke any tools (default when no tools are provided) or to required, which would force our model to use the provided tools for its responses.

Tip 💡: For tasks that require information from a database, complex calculations, domain-specific knowledge, and real-time information (among others), it's good practice to require that the model uses given tools for accurate responses.

Processing the Tool Calls

Now that we've defined our tools, we can process the assistant message and construct the required messages to continue the conversation by invoking each tool call against our get_bakery_prices tool:

[ ]

Finally, we run our final chat completion with multiple tool call results in parallel included in the messages array.

Note: It's best practice to pass the tool definitions again to help the model understand the assistant message with the tool call and to interpret the tool results.

[ ]

By harnessing the power of parallel tool use, you can build more sophisticated and responsive applications that can handle a wide range of complex queries and tasks, providing users with more accurate and timely information.

In general, tool use opens up countless possibilities for efficient, highly-accurate automation. In fact, the innovation in this space is bringing us to a point where we don't even need to build out our own tools. There are startups that provide a marketplace of pre-built tools to use, which you can see in action in our Groq <> Toolhouse AI tutorial and Phidata Mixture of Agents tutorial.