LFM2.5 (1.2B) Text Completion
To run this, press "Runtime" and press "Run all" on a free Tesla T4 Google Colab instance!
To install Unsloth on your local device, follow our guide. This notebook is licensed LGPL-3.0.
You will learn how to do data prep, how to train, how to run the model, & how to save it
News
Train MoEs - DeepSeek, GLM, Qwen and gpt-oss 12x faster with 35% less VRAM. Blog
You can now train embedding models 1.8-3.3x faster with 20% less VRAM. Blog
Ultra Long-Context Reinforcement Learning is here with 7x more context windows! Blog
3x faster LLM training with 30% less VRAM and 500K context. 3x faster • 500K Context
New in Reinforcement Learning: FP8 RL • Vision RL • Standby • gpt-oss RL
Visit our docs for all our model uploads and notebooks.
Installation
Unsloth
Text Completion / Raw Text Training
This is a community notebook collaboration with [Mithex].
We train on Tiny Stories (link here) which is a collection of small stories. For example:
Once upon a time, there was a little car named Beep. Beep loved to go fast and play in the sun.
Beep was a healthy car because he always had good fuel....
Instead of Alpaca's Question Answer format, one only needs 1 column - the "text" column. This means you can finetune on any dataset and let your model act as a text completion model, like for novel writing.
env: UNSLOTH_RETURN_LOGITS=1 # Run this to disable CCE since it is not supported for CPT
==((====))== Unsloth 2025.12.8: Fast Lfm2 patching. Transformers: 4.57.3. \\ /| Tesla T4. Num GPUs = 1. Max memory: 14.741 GB. Platform: Linux. O^O/ \_/ \ Torch: 2.9.0+cu126. CUDA: 7.5. CUDA Toolkit: 12.6. Triton: 3.5.0 \ / Bfloat16 = FALSE. FA [Xformers = 0.0.33.post1. FA2 = False] "-____-" Free license: http://github.com/unslothai/unsloth Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!
We now add LoRA adapters so we only need to update 1 to 10% of all parameters!
We also add embed_tokens and lm_head to allow the model to learn out of distribution data.
/usr/local/lib/python3.12/dist-packages/peft/tuners/tuners_utils.py:916: UserWarning: Model with `tie_word_embeddings=True` and the tied_target_modules=['model.embed_tokens', 'lm_head'] are part of the adapter. This can lead to complications, for example when merging the adapter or converting your model to formats other than safetensors. See for example https://github.com/huggingface/peft/issues/2018. warnings.warn(
Unsloth: Making `model.base_model.model.model.embed_tokens` require gradients
Data Prep
We now use the Tiny Stories dataset from https://huggingface.co/datasets/roneneldan/TinyStories. We only sample the first 5000 rows to speed training up. We must add EOS_TOKEN or tokenizer.eos_token or else the model's generation will go on forever.
Generating train split: 0%| | 0/2119719 [00:00<?, ? examples/s]
Generating validation split: 0%| | 0/21990 [00:00<?, ? examples/s]
Map: 0%| | 0/2500 [00:00<?, ? examples/s]
Print out 5 stories from Tiny Stories
========================= One day, a little girl named Lily found a needle in her room. She knew it was difficult to play with it because it was sharp. Lily wanted to share the needle with her mom, so she could sew a button on her shirt. Lily went to her mom and said, "Mom, I found this needle. Can you share it with me and sew my shirt?" Her mom smiled and said, "Yes, Lily, we can share the needle and fix your shirt." Together, they shared the needle and sewed the button on Lily's shirt. It was not difficult for them because they were sharing and helping each other. After they finished, Lily thanked her mom for sharing the needle and fixing her shirt. They both felt happy because they had shared and worked together.<|im_end|> ========================= Once upon a time, there was a little car named Beep. Beep loved to go fast and play in the sun. Beep was a healthy car because he always had good fuel. Good fuel made Beep happy and strong. One day, Beep was driving in the park when he saw a big tree. The tree had many leaves that were falling. Beep liked how the leaves fall and wanted to play with them. Beep drove under the tree and watched the leaves fall on him. He laughed and beeped his horn. Beep played with the falling leaves all day. When it was time to go home, Beep knew he needed more fuel. He went to the fuel place and got more healthy fuel. Now, Beep was ready to go fast and play again the next day. And Beep lived happily ever after.<|im_end|> ========================= One day, a little fish named Fin was swimming near the shore. He saw a big crab and wanted to be friends. "Hi, I am Fin. Do you want to play?" asked the little fish. The crab looked at Fin and said, "No, I don't want to play. I am cold and I don't feel fine." Fin felt sad but wanted to help the crab feel better. He swam away and thought of a plan. He remembered that the sun could make things warm. So, Fin swam to the top of the water and called to the sun, "Please, sun, help my new friend feel fine and not freeze!" The sun heard Fin's call and shone its warm light on the shore. The crab started to feel better and not so cold. He saw Fin and said, "Thank you, little fish, for making me feel fine. I don't feel like I will freeze now. Let's play together!" And so, Fin and the crab played and became good friends.<|im_end|> ========================= Once upon a time, in a land full of trees, there was a little cherry tree. The cherry tree was very sad because it did not have any friends. All the other trees were big and strong, but the cherry tree was small and weak. The cherry tree was envious of the big trees. One day, the cherry tree felt a tickle in its branches. It was a little spring wind. The wind told the cherry tree not to be sad. The wind said, "You are special because you have sweet cherries that everyone loves." The cherry tree started to feel a little better. As time went on, the cherry tree grew more and more cherries. All the animals in the land came to eat the cherries and play under the cherry tree. The cherry tree was happy because it had many friends now. The cherry tree learned that being different can be a good thing. And they all lived happily ever after.<|im_end|> ========================= Once upon a time, there was a little girl named Lily. Lily liked to pretend she was a popular princess. She lived in a big castle with her best friends, a cat and a dog. One day, while playing in the castle, Lily found a big cobweb. The cobweb was in the way of her fun game. She wanted to get rid of it, but she was scared of the spider that lived there. Lily asked her friends, the cat and the dog, to help her. They all worked together to clean the cobweb. The spider was sad, but it found a new home outside. Lily, the cat, and the dog were happy they could play without the cobweb in the way. And they all lived happily ever after.<|im_end|>
Continued Pretraining
Now let's use Unsloth's UnslothTrainer! More docs here: TRL SFT docs. We do 20 steps to speed things up, but you can set num_train_epochs=1 for a full run, and turn off max_steps=None.
Also set embedding_learning_rate to be a learning rate at least 2x or 10x smaller than learning_rate to make continual pretraining work!
Unsloth: Tokenizing ["text"] (num_proc=4): 0%| | 0/2500 [00:00<?, ? examples/s]
🦥 Unsloth: Padding-free auto-enabled, enabling faster training.
GPU = Tesla T4. Max memory = 14.741 GB. 1.195 GB of memory reserved.
The model is already on multiple devices. Skipping the move to device specified in `args`. ==((====))== Unsloth - 2x faster free finetuning | Num GPUs used = 1 \\ /| Num examples = 2,500 | Num Epochs = 1 | Total steps = 157 O^O/ \_/ \ Batch size per device = 2 | Gradient accumulation steps = 8 \ / Data Parallel GPUs = 1 | Total batch size (2 x 8 x 1) = 16 "-____-" Trainable parameters = 97,517,568 of 1,276,508,928 (7.64% trained)
Unsloth: Will smartly offload gradients to save VRAM!
/usr/local/lib/python3.12/dist-packages/peft/utils/save_and_load.py:279: UserWarning: Setting `save_embedding_layers` to `True` as embedding layers found in `target_modules`.
warnings.warn("Setting `save_embedding_layers` to `True` as embedding layers found in `target_modules`.") 758.4243 seconds used for training. 12.64 minutes used for training. Peak reserved memory = 2.1 GB. Peak reserved memory for training = 0.905 GB. Peak reserved memory % of max memory = 14.246 %. Peak reserved memory for training % of max memory = 6.139 %.
Inference
Let's run the model!
We first will try to see if the model follows the style and understands to write a story that is within the distribution of "Tiny Stories". Ie a story fit for a bed time story most likely.
We select "Once upon a time, in a galaxy, far far away," since it normally is associated with Star Wars.
<|startoftext|>Once upon a time, in a galaxy, far faraway, there was a little girl named Lily. She loved to explore and discover new things. One day, she found a shiny rock in the forest. She was so excited to touch it. Lily asked her mom, "What is this?" Her mom smiled and said, "It's a rock. It's very special. It's like a treasure." Lily was so happy to have found a special rock. She decided to keep it with her forever. She loved to explore and discover new things, and she always remembered the special rock she found in the forest.<|im_end|>
And we're done! If you have any questions on Unsloth, we have a Discord channel! If you find any bugs or want to keep updated with the latest LLM stuff, or need help, join projects etc, feel free to join our Discord!
Some other resources:
- Train your own reasoning model - Llama GRPO notebook Free Colab
- Saving finetunes to Ollama. Free notebook
- Llama 3.2 Vision finetuning - Radiography use case. Free Colab
- See notebooks for DPO, ORPO, Continued pretraining, conversational finetuning and more on our documentation!
