Copyright 2025 Google LLC.

Browser as a tool

LLMs are powerful tools, but are not intrinsically connected to live data sources. Features like Google Search grounding provide fresh information using Google's search index, but to supply truly live information, you can connect a browser to provide up-to-the-minute data and smart exploration.

This notebook will guide you through three examples of using a browser as a tool with the Gemini API, using both the Live Multimodal API and traditional turn-based conversations.

• Requesting live data using a browser tool with the Live API
• Returning images of web pages from function calling
• Connecting to a local network/intranet using a browser tool

Note: for most of the use-cases, you can use the tools directly by Gemini to get it to search content using Google Search, grab videos from YouTube or fetch context from URLs without having to set-up anything. Check out the Grounding notebook for more details.

Set up the SDK

This guide uses the google-genai Python SDK to connect to the Gemini models.

Set up your API key

To run the following cell, your API key must be stored it in a Colab Secret named GOOGLE_API_KEY. If you don't already have an API key, or you're not sure how to create a Colab Secret, see the Authentication quickstart for an example.

Create the SDK client

You will use the same client instance for both the Live API and the classic REST API interactions, so define models for each.

Define some helpers

The show_parts helper renders the deeply nested output that the API returns in an notebook-friendly way; handling text, code and tool calls.

The can_crawl_url helper will perform a robots.txt check to ensure any automated requests are welcome by the remote service.

Browsing live

This example will show you how to use the Multimodal Live API with the Google Search tool, and then comparatively shows a custom web browsing tool to retrieve site contents in real-time.

Use Google Search as a tool

The streaming nature of the Live API requires that the stream processing and function handling code be written in advance. This allows the stream to continue without timing out.

This example uses text as the input mode, and streams text back out, but the technique applies any mode supported by the Live API, including audio.

Now define and run the conversation.

Depending on when you run this, you may note a discrepency between what Google Search has in its index, and what is currently live on Wikipedia. Check out Wikipedia's featured article yourself. Alternatively, the model may decide not to answer due to the requirement for freshness.

To improve this situation, add a browse tool so the model can acquire this information in real-time.

Add a live browser

This step defines a "browser" that requests a URL over HTTP(S), converts the response to markdown and returns it.

This technique works for sites that serve content as full HTML, so sites that rely on scripting to serve content, such as a PWA without SSR, will not work. Check out the visual example later that uses a fully-featured browser.

Now define and run the conversation using the new tool. Here an extended system instruction has been added to coerce the model into calling the tool immediately, so that it doesn't engage in an open-ended conversation that's hard to demonstrate in a notebook.

Browse pages visually

In the previous example, you used a tool to retrieve a page's textual content and use it in a live chat context. However, web pages are a rich multi-modal medium, so using text results in some loss of signal. Using a fully-featured web browser also enables websites that use JavaScript to render content, something that is not possible using a simple HTTP request like the earlier example.

In this example, you will define a tool that takes a screenshot of a web page and passes the image back to the model.

Note: This example automates a headless Chromium browser, so the instructions are specific to a Linux environment and will run on Google Colab. Try this example on Colab, or check out the Selenium documentation for setting up specific browsers in your environment.

Define a graphical browser

Here you define a browse_url function that uses Selenium to load a headless web browser, navigate to a URL and take a screenshot. This technique takes a single screenshot at a fixed size. There are other tools, such as selenium-screenshot, that can capture full-length images by repeatedly scrolling and capturing the page. As this tool is intended for use during a live conversation, this example uses the faster single-shot approach.

Check out the screenshot to make sure it worked.

Connect the browser to the model

Add the browse_url tool to a model and start a chat session. As LLMs do not directly have internet connectivity, modern models like Gemini are trained to tell users that they can't access the internet, rather than hallucinating results. To override this behaviour, this step adds a system instruction that guides the model to use the tool for internet access.

You should see a function_call in the response above. Once the model issues a function call, execute the tool and save both the function_response and the image for the next turn.

If you do not see a function_call, you can either re-run the cell, or continue the chat to answer any questions the model has (e.g. r = chat.send_message('Yes, please use the tool')).

Inspect the image before it is sent back to the model. Depending on where you are running this, you may see localised content. If you are using Google Colab, you can run !curl ipinfo.io to see the geolocation of the running kernal.

Note that if you see a semi-blank image, the page may not have fully loaded. Try adjusting the time.sleep in browse_url, or provide a suitable implementation for the pages you are using in your application.

Browse local services

By providing a browse tool that you run in your own environment, you can connect it to your own private services - such as your home network or intranet.

This example demonstrates how to connect the browse tool to a simulated intranet environment.

First download the sample intranet files.

Set up a HTTP server that serves those files in a background thread, so that you can access it using the main foreground thread.

Set up a host alias to make it look more like a real intranet, and confirm it works.

Take a screenshot to see what the intranet home page looks like.

Finally, start a chat that uses the load_page tool. Include instructions on how to access and navigate the intranet.

Note: If the data you provide to the model is at all sensitive, be sure to read and understand the terms and conditions for the Gemini API, specifically the terms governing how data is processed for paid vs unpaid services.

Further reading

• To learn more about using the grounding tools (search grounding, YouTube links and URL context), check out the Grounding notebook or the one dedicated to Seach grounding.
• For more advanced examples of function calling in the Live API, try the Plotting and Mapping cookbook recipe.

Or browse the Gemini API cookbook.