Notebooks
N
Nebius Guided Json

Guided Json

nebius-token-factory-cookbook
alph-notebooks/nebius-token-factory-cookbook / guided_json.ipynb

Guided JSON output

Open In Colab

Prerequisites

1 - Setup

1.1 - If running on Google Colab

Add NEBIUS_API_KEY to Secrets

1.2 - If running locally

Create an .env file with both keys:

NEBIUS_API_KEY=your_nebius_api_key_here

2 - Install Dependencies

[10]
NOT running on Colab
[11]

3 - Load Configuration

[12]
✅ NEBIUS_API_KEY found

4 - Define Schema

Person

  • name : string
  • age : int
  • occupation : string
[13]
{
  "properties": {
    "name": {
      "description": "The person's full name",
      "title": "Name",
      "type": "string"
    },
    "age": {
      "anyOf": [
        {
          "type": "integer"
        },
        {
          "type": "null"
        }
      ],
      "description": "The person's age if mentioned",
      "title": "Age"
    },
    "occupation": {
      "description": "The person's job or occupation",
      "title": "Occupation",
      "type": "string"
    }
  },
  "required": [
    "name",
    "age",
    "occupation"
  ],
  "title": "PersonInfo",
  "type": "object"
}

5 - Initialize API

We need a model that supports function calling.

  1. Go to models tab in tokenfactory.nebius.com
  2. Select text to text models
  3. Copy the model name
[14]

6 - API Call

Note the following

  1. we are passing schema in system prompt

  2. and specifying guided_json on api call

    extra_body={
        "guided_json": schema,
    },
  1. Also asking for JSON output
response_format={"type": "json_object"},  # Forces JSON output

And note how completion.choices[0].message.content is in JSON format

[ ]
{
  "id": "chatcmpl-b20676a0fba1bcb3",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null,
      "message": {
        "content": "{\"name\":\"Mikhail Kashirskii\",\"age\":34,\"occupation\":\"software developer\"}",
        "refusal": null,
        "role": "assistant",
        "annotations": null,
        "audio": null,
        "function_call": null,
        "tool_calls": [],
        "reasoning": "We need to extract name, age, occupation. The text: \"The task of json generation was assigned Mikhail Kashirskii,  a a 34-year-old software developer at Nebius. His role in this task is to implement the support of json outputs of the LLM.\"\n\nName: Mikhail Kashirskii. Age: 34. Occupation: software developer. Output JSON with required fields. Ensure age integer.",
        "reasoning_content": "We need to extract name, age, occupation. The text: \"The task of json generation was assigned Mikhail Kashirskii,  a a 34-year-old software developer at Nebius. His role in this task is to implement the support of json outputs of the LLM.\"\n\nName: Mikhail Kashirskii. Age: 34. Occupation: software developer. Output JSON with required fields. Ensure age integer."
      },
      "stop_reason": null,
      "token_ids": null
    }
  ],
  "created": 1773208203,
  "model": "openai/gpt-oss-20b",
  "object": "chat.completion",
  "service_tier": null,
  "system_fingerprint": null,
  "usage": {
    "completion_tokens": 115,
    "prompt_tokens": 272,
    "total_tokens": 387,
    "completion_tokens_details": null,
    "prompt_tokens_details": null
  },
  "prompt_logprobs": null,
  "prompt_token_ids": null,
  "kv_transfer_params": null
}

7 - Convert the output to PersonInfo object

[16]
{
  "name": "Mikhail Kashirskii",
  "age": 34,
  "occupation": "software developer"
}

8 - Age Optional

In the resulting PersonInfo, age attribute will be null.

[17]
{
  "id": "chatcmpl-930bc56c5ee3f580",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null,
      "message": {
        "content": "{\"name\":\"Mikhail Kashirskii\",\"age\":null,\"occupation\":\"software developer at Nebius\"}",
        "refusal": null,
        "role": "assistant",
        "annotations": null,
        "audio": null,
        "function_call": null,
        "tool_calls": [],
        "reasoning": "We need to extract name, age, occupation. The text: \"The task of json generation was assigned Mikhail Kashirskii, a software developer at Nebius. His role in this task is to implement the support of json outputs of the LLM.\"\n\nName: Mikhail Kashirskii. Occupation: software developer at Nebius. Age: not mentioned. So age null. Output JSON with name, age null, occupation string. Ensure schema: name string, age integer or null, occupation string. Provide JSON.",
        "reasoning_content": "We need to extract name, age, occupation. The text: \"The task of json generation was assigned Mikhail Kashirskii, a software developer at Nebius. His role in this task is to implement the support of json outputs of the LLM.\"\n\nName: Mikhail Kashirskii. Occupation: software developer at Nebius. Age: not mentioned. So age null. Output JSON with name, age null, occupation string. Ensure schema: name string, age integer or null, occupation string. Provide JSON."
      },
      "stop_reason": null,
      "token_ids": null
    }
  ],
  "created": 1773208204,
  "model": "openai/gpt-oss-20b",
  "object": "chat.completion",
  "service_tier": null,
  "system_fingerprint": null,
  "usage": {
    "completion_tokens": 137,
    "prompt_tokens": 266,
    "total_tokens": 403,
    "completion_tokens_details": null,
    "prompt_tokens_details": null
  },
  "prompt_logprobs": null,
  "prompt_token_ids": null,
  "kv_transfer_params": null
}
[18]
{
  "name": "Mikhail Kashirskii",
  "age": null,
  "occupation": "software developer at Nebius"
}