Fine-Tuning Large Language Models

What is Fine-Tuning?

Fine-tuning is the process of taking a pre-trained large language model and further training it on a smaller, domain-specific dataset to adapt it for particular tasks. Think of it as teaching an experienced generalist to become a specialist.

While prompt engineering works for many use cases, fine-tuning is essential when you need: consistent output formatting, domain-specific knowledge, reduced latency (shorter prompts), or behavior that's difficult to achieve through prompting alone.

Fine-Tuning Methods

Code Example — LoRA Fine-Tuning with Unsloth

# pip install unsloth
from unsloth import FastLanguageModel
import torch

# Load base model with 4-bit quantization
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Llama-3.2-3B-Instruct",
    max_seq_length=2048,
    load_in_4bit=True
)

# Add LoRA adapters
model = FastLanguageModel.get_peft_model(
    model, r=16, lora_alpha=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
    lora_dropout=0
)

# Train with your dataset (SFTTrainer from trl)
from trl import SFTTrainer
from transformers import TrainingArguments

trainer = SFTTrainer(
    model=model,
    train_dataset=dataset,
    args=TrainingArguments(
        output_dir="./output",
        per_device_train_batch_size=2,
        num_train_epochs=3,
        learning_rate=2e-4
    )
)
trainer.train()
model.save_pretrained("my-fine-tuned-model")

Dataset Preparation

• Instruction Format: Use chat template format with system, user, and assistant messages.
• Quality > Quantity: 1,000 high-quality examples often outperform 100,000 poor ones.
• Diversity: Cover edge cases, different phrasings, and error scenarios.
• Validation Set: Always reserve 10-20% of data for evaluation.