Notebooks
W
Weights and Biases LLM Finetuning Notebook

LLM Finetuning Notebook

wandb-exampleshuggingfacecolabs
alph-notebooks/wandb-examples / LLM_Finetuning_Notebook.ipynb

Open In Colab

LLM Finetuning with HuggingFace and Weights and Biases

  • Fine-tune a lightweight LLM (OPT-125M) with LoRA and 8-bit quantization using Launch
  • Checkpoint the LoRA adapter weights as artifacts
  • Link the best checkpoint in Model Registry
  • Run inference on a quantized model

The same workflow and principles from this notebook can be applied to fine-tuning some of the stronger OSS LLMs (e.g. Llama2)

Fine-tune large models using 🤗 peft adapters, transformers & bitsandbytes

In this tutorial we will cover how we can fine-tune large language models using the very recent peft library and bitsandbytes for loading large models in 8-bit. The fine-tuning method will rely on a recent method called "Low Rank Adapters" (LoRA), instead of fine-tuning the entire model you just have to fine-tune these adapters and load them properly inside the model. After fine-tuning the model you can also share your adapters on the 🤗 Hub and load them very easily. Let's get started!

Install requirements

First, run the cells below to install the requirements:

[ ]

Model Loading

  • Here we leverage 8-bit quantization to reduce the memory footprint of the model during training
[ ]

Post-processing on the model

Finally, we need to apply some post-processing on the 8-bit model to enable training, let's freeze all our layers, and cast the layer-norm in float32 for stability. We also cast the output of the last layer in float32 for the same reasons.

[ ]

Apply LoRA

Here comes the magic with peft! Let's load a PeftModel and specify that we are going to use low-rank adapters (LoRA) using get_peft_model utility function from peft.

[ ]
[ ]
[ ]

Training

  • W&B HuggingFace integration automatically tracks important metrics during the course of training
  • Also track the HF checkpoints as artifacts and register them in the model registry!
  • Change the number of steps to 200+ for real results!
[ ]
project_name
entity

Adding Model Weights to W&B Model Registry

  • Here we get our best checkpoint from the finetuning run and register it as our best model
[ ]
last_run_id
registered_model_name

Consuming Model From Registry and Quantizing using ctranslate2

  • LLMs are typically too large to run in full-precision on even decent hardware.
  • You can quantize the model to run it more efficiently with minimal loss in accuracy.
    • CTranslate2 is a great first pass at quantization but doesn't do "smart" quantization. It just converts all weights to half precision.
    • Checkout out GPTQ and AutoGPTQ for SOTA quantization at scale
[ ]
[ ]
[ ]

Run Inference Using Quantized CTranslate2 Model

  • Record the results in a W&B Table!
[ ]