Batch Mode
quickstartsgemini-cookbookgemini-apigemini
Export
Copyright 2025 Google LLC.
[ ]
Gemini Batch API
The Gemini Batch API is designed for processing large volumes of non-latency-critical requests asynchronously. It's ideal for workloads that require high throughput, such as pre-processing datasets, running large-scale evaluations, or generating content in bulk.
Key Benefits:
- High throughput: Process millions of requests in a single job.
- Cost savings: Batches are priced at a 50% discount compared to the standard API.
- Asynchronous: Submit your job and retrieve the results later, within a 24-hour SLO.
In this notebook, you will learn how to:
- Set up your environment for Batch Mode.
- Create a batch job by uploading a JSONL file (recommended for large jobs).
- Create a batch job using inline requests (convenient for smaller jobs).
- Monitor the status of your job.
- Retrieve and parse the results for both job types.
- Manage your jobs (list and cancel).
- Use batch embeddings.
- Generate with multimodal inputs.
- Generate images.
Setup
Install SDK
Install the SDK from PyPI.
[1]
Setup your API key
To run the following cell, your API key must be stored it in a Colab Secret named GOOGLE_API_KEY. If you don't already have an API key, or you're not sure how to create a Colab Secret, see Authentication 
for an example.
[2]
Initialize SDK client
Initialize a client with your API key:
[3]
Choose a model
Most Gemini models are compatible with Batch mode, but the size of the queue is not the same for each of them. Refer to the documentation for more details.
[4]
MODEL_ID
Media gen models (Imagen, Lyria or Veo) are not currently compatible with Batch API. But if you want to batch create images you can use the Nano-Banana model (see example below).
Creating Batch Jobs: Two Methods
You can create batch jobs in two ways:
- File-based (
src): Upload a JSONL file containing all your requests. This is the recommended method for large datasets. - Inline (
inlined_requests): Pass a list of request objects directly in your code. This is convenient for smaller, dynamically generated jobs.
Create a job from a file
This is the most common workflow. You will prepare an input file, upload it, create the job, monitor it, and retrieve the results.
Step 1: Prepare and upload the input File
The input file must be a JSON file, where each line is a JSON object. Each object must contain a unique key to help you correlate inputs with outputs, and a request object matching the GenerateContentRequest schema.
[ ]
Uploading file: batch_requests.json Uploaded file: files/p1mds9fl15bs
Step 2: Create the Batch Job
Now, pass the uploaded file's name (uploaded_batch_requests.name) to the client.batches.create function to create the batch job.
[ ]
Created batch job from file: batches/o08hk76gv328ihxcssjsgwt3g6omtfysxi46
Step 3: Monitor job status
Jobs can take time to complete (up to 24 hours). You can poll the API to check the status.
[ ]
Polling status for job: batches/o08hk76gv328ihxcssjsgwt3g6omtfysxi46 Job not finished. Current state: JOB_STATE_PENDING. Waiting 30 seconds... Job not finished. Current state: JOB_STATE_PENDING. Waiting 30 seconds... Job not finished. Current state: JOB_STATE_PENDING. Waiting 30 seconds... Job not finished. Current state: JOB_STATE_PENDING. Waiting 30 seconds... Job not finished. Current state: JOB_STATE_PENDING. Waiting 30 seconds... Job not finished. Current state: JOB_STATE_PENDING. Waiting 30 seconds... Job not finished. Current state: JOB_STATE_PENDING. Waiting 30 seconds... Job not finished. Current state: JOB_STATE_PENDING. Waiting 30 seconds... Job not finished. Current state: JOB_STATE_PENDING. Waiting 30 seconds... Job finished with state: JOB_STATE_SUCCEEDED
Step 4: Retrieve and parse results
Once a file-based job succeeds, the results are written to an output file in the Files API.
[ ]
Results are in file: files/batch-o08hk76gv328ihxcssjsgwt3g6omtfysxi46
Downloading and parsing result file content...
{
"response": {
"responseId": "6VFkaIDwEPXVjMcP4-XW6Qo",
"usageMetadata": {
"totalTokenCount": 848,
"promptTokensDetails": [
{
"modality": "TEXT",
"tokenCount": 8
}
],
"thoughtsTokenCount": 829,
"candidatesTokenCount": 11,
"promptTokenCount": 8
},
"candidates": [
{
"content": {
"role": "model",
"parts": [
{
"text": "AI learns patterns from data to make decisions or predictions."
}
]
},
"finishReason": "STOP",
"index": 0
}
],
"modelVersion": "gemini-2.5-flash"
},
"key": "request_1"
}
--------------------
{
"response": {
"modelVersion": "gemini-2.5-flash",
"candidates": [
{
"index": 0,
"finishReason": "STOP",
"content": {
"role": "model",
"parts": [
{
"text": "Quantum computers use **qubits** that can be 0, 1, or both simultaneously (**superposition**), and be interconnected (**entanglement**). They manipulate these states, causing correct answers to **reinforce** and incorrect ones to cancel out, to solve complex problems by exploring vast possibilities at once."
}
]
}
}
],
"responseId": "7FFkaJK7EaWOjMcP-LvX2AQ",
"usageMetadata": {
"promptTokensDetails": [
{
"modality": "TEXT",
"tokenCount": 9
}
],
"candidatesTokenCount": 63,
"promptTokenCount": 9,
"thoughtsTokenCount": 1180,
"totalTokenCount": 1252
}
},
"key": "request_2"
}
--------------------
Alternative: Create a job with inline requests
For smaller tasks, you can pass requests directly without creating a file. The results will be returned directly in the job object itself.
Step 1: Create and monitor the inline job
[ ]
Creating inline batch job... Created inline batch job: batches/mqnrj9gul3irzwwy4vgawrv9w40vxbvou6x2 -------------------- Polling status for job: batches/mqnrj9gul3irzwwy4vgawrv9w40vxbvou6x2 Job not finished. Current state: JOB_STATE_SUCCEEDED. Waiting 30 seconds... Job not finished. Current state: JOB_STATE_SUCCEEDED. Waiting 30 seconds... Job not finished. Current state: JOB_STATE_SUCCEEDED. Waiting 30 seconds... Job not finished. Current state: JOB_STATE_SUCCEEDED. Waiting 30 seconds... Job not finished. Current state: JOB_STATE_SUCCEEDED. Waiting 30 seconds... Job not finished. Current state: JOB_STATE_SUCCEEDED. Waiting 30 seconds... Job not finished. Current state: JOB_STATE_SUCCEEDED. Waiting 30 seconds... Job not finished. Current state: JOB_STATE_SUCCEEDED. Waiting 30 seconds... Job not finished. Current state: JOB_STATE_SUCCEEDED. Waiting 30 seconds... Job not finished. Current state: JOB_STATE_SUCCEEDED. Waiting 30 seconds... Job not finished. Current state: JOB_STATE_SUCCEEDED. Waiting 30 seconds... Job finished with state: JOB_STATE_SUCCEEDED
Step 2: Retrieve and print inline results
Once the job has succeeded, the results are available in the inlined_responses field of the job object. You can iterate through this list to get each response.
[ ]
Results are inline: --- Response 1 --- A fluffy, soft white sight, It drifts with gentle light. Across the sky so blue, A changing, dreamy view. --- Response 2 --- A silent step, a velvet paw, He naps, ignoring every law. A sudden stretch, a sleepy blink, Then purrs, much deeper than you think.
Managing jobs
Here are some common operations for managing your batch jobs.
List your batch jobs
List Recent Jobs
[ ]
Listing recent batch jobs: Job Name: batches/mqnrj9gul3irzwwy4vgawrv9w40vxbvou6x2 - Display Name: my-batch-job-inline-example - State: JOB_STATE_SUCCEEDED - Create Time: 2025-07-01 21:24:08 -------------------- Job Name: batches/o08hk76gv328ihxcssjsgwt3g6omtfysxi46 - Display Name: my-batch-job-from-file - State: JOB_STATE_SUCCEEDED - Create Time: 2025-07-01 21:19:34 - Type: File-based (Output: files/batch-o08hk76gv328ihxcssjsgwt3g6omtfysxi46) -------------------- Job Name: batches/7y78ioanoiaatpx7skou4bj9qo4xqv97p2cf - Display Name: my-batch-image-gen-job - State: JOB_STATE_SUCCEEDED - Create Time: 2025-07-01 20:50:48 - Type: File-based (Output: files/batch-7y78ioanoiaatpx7skou4bj9qo4xqv97p2cf) -------------------- Job Name: batches/pd6qbrq4izis2xzc653uczycn2wu2at6hdw4 - Display Name: my-batch-job-with-image - State: JOB_STATE_SUCCEEDED - Create Time: 2025-07-01 20:50:43 - Type: File-based (Output: files/batch-pd6qbrq4izis2xzc653uczycn2wu2at6hdw4) -------------------- Job Name: batches/8wnfd6mvbat85cshphnfi6pbnh6qhtcxolhx - Display Name: my-batch-image-gen-job - State: JOB_STATE_SUCCEEDED - Create Time: 2025-07-01 20:45:32 - Type: File-based (Output: files/batch-8wnfd6mvbat85cshphnfi6pbnh6qhtcxolhx) -------------------- Job Name: batches/bs606yinq47k4eu2t6a72r1te3umwlkwj3zr - Display Name: my-batch-job-with-image - State: JOB_STATE_SUCCEEDED - Create Time: 2025-07-01 20:45:26 - Type: File-based (Output: files/batch-bs606yinq47k4eu2t6a72r1te3umwlkwj3zr) -------------------- Job Name: batches/rbor72jehk6a8zk7al9rcwn29n707n9rijvb - Display Name: my-batch-job-inline-example - State: JOB_STATE_SUCCEEDED - Create Time: 2025-07-01 20:32:16 -------------------- Job Name: batches/ypd4nvghou6dpwlgzgjwrgfcs879m5jp2lq6 - Display Name: my-batch-job-from-file - State: JOB_STATE_SUCCEEDED - Create Time: 2025-07-01 20:29:19 - Type: File-based (Output: files/batch-ypd4nvghou6dpwlgzgjwrgfcs879m5jp2lq6) -------------------- Job Name: batches/1t1dldu0lqx05a1sqmhhe0r1l554fborc04d - Display Name: my-batch-job-with-image - State: JOB_STATE_SUCCEEDED - Create Time: 2025-06-20 15:16:33 - Type: File-based (Output: files/batch-1t1dldu0lqx05a1sqmhhe0r1l554fborc04d) -------------------- Job Name: batches/vqtwzzbnbj0gbwsiv2e2xkk1pzv9lvyfsurs - Display Name: my-batch-job-inline-example - State: JOB_STATE_SUCCEEDED - Create Time: 2025-06-20 15:13:17 --------------------
Cancel a Batch Job (Optional)
If you need to stop a job that is still pending or running, you can cancel it.
Cancel a Job
The next cell is commented out to prevent accidental cancellation.
To use it:
- Get the name of the job you want to cancel (e.g., from the list above).
- Uncomment the code and replace the placeholder name.
[ ]
Batch embeddings
You can use the Batch API to interact with the Embeddings model for higher throughput. When creating a batch job, specify the embeddings model, and set the source to either inlined_requests for inline requests or file_name for input files.
In this example, create a JSONL file:
[ ]
Writing embeddings_batch_requests.jsonl
Upload the file:
[ ]
File( , create_time=datetime.datetime(2025, 9, 10, 13, 37, 39, 328452, tzinfo=TzInfo(UTC)), , expiration_time=datetime.datetime(2025, 9, 12, 13, 37, 39, 53649, tzinfo=TzInfo(UTC)), , mime_type='jsonl', , name='files/t5snrshrqff1', , sha256_hash='NTk3Yjk2ZTVjN2QyYjBkMDFhY2RjODBkZDBiY2Q2NTNkYTA5OWJiMjZkODEwNmIxMmQ4NzBlZGFkZjI4MzdmYw==', , size_bytes=202, , source=<FileSource.UPLOADED: 'UPLOADED'>, , state=<FileState.ACTIVE: 'ACTIVE'>, , update_time=datetime.datetime(2025, 9, 10, 13, 37, 39, 328452, tzinfo=TzInfo(UTC)), , uri='https://generativelanguage.googleapis.com/v1beta/files/t5snrshrqff1' ,)
Submit batch embeddings job
[ ]
/var/folders/nx/bjv94y0x69jf7tlvxwxwcbvm01b332/T/ipykernel_20871/20559616.py:1: ExperimentalWarning: batches.create_embeddings() is experimental and may change without notice. batch_job = client.batches.create_embeddings(
BatchJob( , create_time=datetime.datetime(2025, 9, 10, 14, 42, 7, 160573, tzinfo=TzInfo(UTC)), , model='models/gemini-embedding-001', , name='batches/2icf9ky95pieco1jwucta7drijo5k8hjllzj', , state=<JobState.JOB_STATE_PENDING: 'JOB_STATE_PENDING'>, , update_time=datetime.datetime(2025, 9, 10, 14, 42, 7, 160573, tzinfo=TzInfo(UTC)) ,)
Check batch state and wait up to 24h:
[ ]
Job not finished. Current state: JOB_STATE_PENDING. Waiting 30 seconds... Job not finished. Current state: JOB_STATE_PENDING. Waiting 30 seconds...
Download and print the embeddings results:
[ ]
{"response":{"tokenCount":"1","embedding":{"values":[-0.034119852,0.0025719088,0.017325249]}},"key":"request_1"}
{"response":{"tokenCount":"1","embedding":{"values":[0.020208105,-0.016668973,-0.011120025,-0.063684635]}},"key":"request_2"}
Other examples
Using multimodal input
Here's an example using multimodal input. Once again it will use the Files API to store the image you want to send along with your prompts.
[ ]
Uploading image file: jetpack.jpg Uploaded image file: files/sm3dyq4g4zqy with MIME type: image/jpeg
[ ]
Creating JSONL file: batch_requests_with_image.json Uploading JSONL file: batch_requests_with_image.json Uploaded JSONL file: files/hq429pdc5pfk Creating batch job... Created batch job from file: batches/3ar0jhcayt8o9gixe3nv22xdsxooh0ednsi6 You can now monitor the job status using its name.
[ ]
Results are in file: files/batch-3ar0jhcayt8o9gixe3nv22xdsxooh0ednsi6
Downloading and parsing result file content...
{
"parts": [
{
"text": "AI learns from data to identify patterns and make decisions."
}
],
"role": "model"
}
--------------------
{
"role": "model",
"parts": [
{
"text": "The image displays a detailed, hand-drawn conceptual sketch of a \"JETPACK BACKPACK\" on a sheet of white paper with blue horizontal lines, typical of ruled notebook paper. All elements, including the drawing and text, are rendered in blue ink, giving it the appearance of a casual brainstorming or design note.\n\nAt the top of the page, the title **\"JETPACK BACKPACK\"** is prominently displayed, underlined and written in bold capital letters.\n\nThe central feature is a stylized drawing of a backpack, depicted from a slightly elevated, possibly three-quarter, angle, showing its top and front/sides. It has a rounded, somewhat oval shape, wider at the top and tapering slightly towards the bottom. The front of the backpack shows a main compartment or structural lines.\n\nSeveral annotated labels, connected by arrows to specific parts of the drawing or general areas, describe the features and specifications of this conceptual device:\n\n* **Top (pointing to the top of the backpack):** \"FITS 18\" LAPTOP\" \u2013 indicating its capacity for a large laptop.\n* **Top Left (pointing to the top-side of the backpack):** \"PADDED STRAP SUPPORT\" \u2013 suggesting comfort for the wearer.\n* **Middle Right (pointing to the body of the backpack):** Two labels describe its general characteristics: \"LIGHTWEIGHT,\" and \"LOOKS LIKE A NORMAL BACKPACK\" \u2013 emphasizing its discreet and practical design.\n* **Bottom Right (pointing to the exhaust nozzles):** \"RETRACTABLE BOOSTERS\" \u2013 describing the mechanism for the jetpack function.\n* **Bottom Right (pointing to the steam plume):** \"STEAM-POWERED, GREEN/CLEAN\" \u2013 specifying the propulsion method and its environmental benefits.\n* **Bottom Left (pointing towards the bottom of the backpack):** \"USB-C CHARGING\" \u2013 indicating the method for recharging the device.\n* **Bottom Left (general note):** \"15-MIN BATTERY LIFE\" \u2013 providing a key performance specification.\n\nEmanating downwards from the bottom of the backpack, where the boosters are indicated, are two large, voluminous, swirling plumes. These are depicted with multiple crescent and spiral shapes, visually representing steam or exhaust, consistent with the \"STEAM-POWERED\" annotation.\n\nThe overall impression is that of an initial design concept or an idea pitch for a personal flight device that is designed to be highly portable, discreet (looking like a normal backpack), environmentally friendly, and equipped with modern charging capabilities, albeit with a limited flight duration."
}
]
}
--------------------
Multimodal output
This time you're going to batch generate images using the Nano-Banana model. More details on the dedicated notebook.
[6]
Creating JSONL file: batch_image_gen_requests.json Uploading JSONL file: batch_image_gen_requests.json Uploaded JSONL file: files/4k1cfn983f0a Creating batch job... Created batch job from file: batches/6czrv8tda2rbsvn7gno27npxly1pyv5xjjh2 You can now monitor the job status using its name.
[8]
The output could not be saved in the notebook but here's what it could look like:
Next Steps
Useful API references:
For more details on the Batch API, please check the related documentation.
Process your collected log datasets using the Batch API
Check out this advanced use-case using the batch API to process log datasets.
Continue your discovery of the Gemini API
Check other nice Gemini capabilities in the Cookbook. In particular, the quickstarts folder is full of guides on how to use the Gemini (and gen-media) models and features, while the example folder showcase cool use-cases to spark your creativity.