LLM Fine-tuning cookbook

Everyone argues fine-tuning versus RAG, but which one does my actual, narrow task need?

No idea what format a trainer wants.

I've read the LoRA theory five times and still haven't run one fine-tune of my own.

I copied rank 16, alpha 32 from a tutorial without knowing if those numbers even fit my task.

Everyone tells me to use LoRA or QLoRA, but I don't know which one fits my GPU, or whether full fine-tuning is worth the extra cost.

My fine-tuning script died with a CUDA out of memory error. Dropping the batch size to 1 didn't fix it.

My fine-tuned model nails every question I test it with, and that success is what worries me instead of reassuring me.

After fine-tuning on support replies, my model started giving one-word answers to questions that had nothing to do with support.

My LoRA adapter works fine in testing, but the server I need to deploy to only accepts one plain model file.

My training script died overnight and I don't know if resuming from the checkpoint will pick up cleanly or quietly start over.

My training loss looks great, but the model just repeats one answer no matter what I ask it.

The same handful of support tickets, copied into my training set a dozen times over, and I never noticed.

My fine-tuned model gets the facts right and still sounds wrong for our brand, and I don't know how to train that away.

My fine-tuning loss just sits flat, or spikes randomly, and I don't know if it's the learning rate, the data, or something else entirely.

I fine-tuned one adapter for tone and another for a skill, and now I need both without training a third model from scratch.