LLM Fine-Tuning Decision Guide: When to Fine-Tune, When to Prompt, and When to Use RAG Instead
Teams often jump to fine-tuning when simpler approaches would suffice. Fine-tuning is the right choice only in specific scenarios.
1. The Decision Framework
| Scenario | Best Approach | Why |
|---|---|---|
| Model lacks domain knowledge | RAG | Cheaper than fine-tuning, knowledge stays current |
| Model knows the info but outputs wrong format | Prompt Engineering | Few-shot examples fix formatting |
| Need consistent brand voice/personality | Fine-Tuning | Bakes style into model weights |
| Task requires specialized reasoning patterns | Fine-Tuning | Teaches new reasoning strategies |
| Knowledge changes frequently | RAG | Fine-tuned knowledge goes stale |
2. QLoRA Fine-Tuning Example
from unsloth import FastLanguageModel
from trl import SFTTrainer
model, tokenizer = FastLanguageModel.from_pretrained(
"meta-llama/Llama-3.3-70B",
max_seq_length=4096,
load_in_4bit=True # QLoRA: 4-bit quantized base + trainable LoRA adapters
)
model = FastLanguageModel.get_peft_model(model, r=16, lora_alpha=32)
trainer = SFTTrainer(
model=model,
train_dataset=dataset,
max_seq_length=4096,
dataset_text_field="text",
args=TrainingArguments(
per_device_train_batch_size=4,
gradient_accumulation_steps=4,
num_train_epochs=3,
learning_rate=2e-4,
output_dir="./fine-tuned-model"
)
)
trainer.train()
Fine-tuning is a precision tool — powerful when applied correctly, wasteful when a simpler approach would achieve the same result.
