LLM Finetuning Cookbook with NeMo AutoModel

This notebook is a finetuning recipe that walks through:

1. Environment setup — Use the existing repo and environment at ~/Automodel.
2. Dataset creation — Build training, validation, and test sets in the JSONL format expected by the finetuning script (input / output fields).
3. Finetuning — Run PEFT (LoRA) finetuning using examples/llm_finetune/finetune.py and examples/llm_finetune/nemotron/base-peft-config-cookbook.yaml.

Requirements: Multi-GPU machine (e.g. 8× H100 for Nemotron-3-Super.

1. Environment setup

1.1 Set up the Workspace ROOT and Data Paths

1.2 Set up the Working Directory

1.3 Set up the Working Environment

Optional (Nemotron-Super / Mamba): If you use the Nemotron-Super checkpoint and hit ABI issues with mamba-ssm, install it with --no-build-isolation so it is built against your venv's PyTorch:

cd "$AUTOMODEL_DIR" && source .venv/bin/activate
pip install mamba-ssm --no-build-isolation

Hugging Face: For nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16), run huggingface-cli login in the same environment before finetuning.

2. Dataset creation

In this cookbook, the BIRD-SQL dataset is used for fine-tuning Nemotron-3-super with NeMo AutoModel.

A Little about the dataset:

BIRD-SQL is a large-scale text-to-SQL benchmark that focuses on real-world conditions: it has 12,751 question–SQL pairs over 95 databases from 37 domains (about 33.4 GB total). Unlike benchmarks that only stress schema understanding, BIRD-SQL uses realistic, “dirty” data and requires reasoning about database contents and external knowledge, making it a good fit for training models for practical text-to-SQL.

The cells below load the BIRD-SQL, convert it to input/output JSONL, and write training.jsonl, validation.jsonl, and test.jsonl to DATASET_DIR.

The config in the next cell controls the split: VALIDATION_FRACTION and TEST_FRACTION set aside validation and test data, SPLIT_SEED fixes reproducibility, INCLUDE_REASONING toggles reasoning text (vs. BIRD-only, like --no-reasoning), BIRD_ONLY restricts the source to xu3kev/BIRD-SQL-data-train, and LIMIT caps the number of examples.

3. Finetuning

We run the recipe script from the AutoModel repo root:

torchrun --nproc-per-node=N examples/llm_finetune/finetune.py --config examples/llm_finetune/nemotron/base-peft-config-cookbook.yaml

Make sure you copy the config file (exists at the same place as this notebook, on Nemotron Github) to examples/llm_finetune/nemotron and the dataset target file to nemo_automodel/components/datasets/llm within the cloned Automodel directory before running the following cells.

4. Deploy: load base + PEFT and run inference

After finetuning, load the base model and PEFT adapter (from the checkpoint resolved by the cell above) and run inference with transformers + peft. No merge step required — the adapter is applied at load time.

Summary

• Environment: Use the existing repo and env at AUTOMODEL_DIR (no clone or new venv).
• Data: Create training.jsonl, validation.jsonl, and test.jsonl in DATASET_DIR with input/output (or your recipe’s expected format).
• Finetuning: From the AutoModel repo root, run torchrun --nproc-per-node=N examples/llm_finetune/finetune.py --config examples/llm_finetune/nemotron/base-peft-config-cookbook.yaml, overriding dataset.path_or_dataset and model.pretrained_model_name_or_path as needed.

For Nemotron-3-Super, use a config with the correct LoRA targets and a smaller local_batch_size / seq_length to avoid OOM; enable activation_checkpointing if supported.

Step 6: Deploying the Trained Model for Inference

You can deploy the model you just trained by following the cookbook under usage-cookbook/Nemotron-3-Super/vllm_cookbook.ipynb and providing the local path of the merged checkpoint (i.e. the path stored in MERGE_OUTPUT_DIR) as the model_id.