The Fine-Tuning Guide I Trust in 2026
Fine-tuning changes what a model does, not what it knows. A practical fine tuning guide to LoRA, QLoRA, SFT, and DPO, and the evals that catch what breaks.
Fine-tuning changes what a model does, not what it knows. A practical fine tuning guide to LoRA, QLoRA, SFT, and DPO, and the evals that catch what breaks.
A fine tuning guide should start with what fine-tuning is not. It does not teach a model new facts. It changes what the model does with the facts it already has: its format, its tone, its default way of handling a task it sees over and over. If you want a model to know something new, that is a retrieval problem. If you want a model to behave a specific way every time, that is what fine-tuning is for.
In 2026 the decision comes down to two forks: which method (SFT, DPO, or RFT) and where you run it (a narrowing set of hosted provider platforms, or an open-weight model you fine-tune yourself with LoRA or QLoRA). Neither fork is about the learning rate. Both are about what you are willing to depend on, and what you are willing to test before you trust the result.
Key takeaways
- Fine-tuning changes behavior, not knowledge. It reshapes format, tone, and task reliability. It does not durably add facts a model did not already have.
- LoRA and QLoRA are the default in 2026. Full fine-tuning is now the exception, reserved for teams with the budget and the reason to want it.
- QLoRA fits a 65B model on one 48GB GPU while matching full 16-bit fine-tuning quality, according to the method's original paper.
- OpenAI is winding down its hosted fine-tuning platform for new users in 2026, which pushes the durable default toward open-weight models you control.
- The dataset matters more than the method. A few hundred clean, representative examples beat ten thousand noisy ones, and a held-out set is not optional.
What fine-tuning actually changes (and what it doesn't)
Start with a definition you can act on. Fine-tuning takes a pretrained model and continues training it on a smaller, task-specific dataset, adjusting its weights, or a small adapter layered on top, so its default outputs shift toward the examples you showed it. The model's underlying knowledge stays mostly where it was. What moves is the shape of the answer: the format it defaults to, the tone it writes in, how it handles a request it has now seen thousands of times.
That distinction is the single most common reason fine-tuning projects fail. A team ships a fine-tune to fix a model that keeps citing an old price, and it does nothing durable, because the problem was never behavior. The facts changed and the model's frozen weights did not know it. Fine-tuning cannot fix a freshness problem. Only retrieval can, because retrieval reads from a store you update, not from weights you trained once.
The reverse mistake happens too: teams pile more documents into a RAG pipeline to fix a model that writes in the wrong voice, and the extra context does not help, because the problem was never missing knowledge. It was behavior that needed to be trained in, not looked up. For the fuller version of that seam, see my RAG vs fine-tuning comparison. Here I am assuming you already decided fine-tuning is the tool.
Fine-tuning vs. prompting vs. RAG: how to decide
Before you fine-tune anything, rule out the cheaper options. Prompting is the fastest lever: a longer, better-structured system prompt often gets the behavior you want with no training run. Retrieval is the second lever: if the model is missing facts, feed it the facts at query time. Fine-tuning is the last lever, reached for when behavior must hold under load or the prompt describing it grows too long and fragile.
Ask this before every fine-tuning project: if you could not touch the model's weights, could a better prompt or better retrieved context fix this? If yes, do that first. It is cheaper, faster to iterate, and does not commit you to a training pipeline. Fine-tune only the residual problem prompting and retrieval cannot touch.
The fine-tuning methods that matter in 2026: SFT, DPO, and RFT
Three methods cover almost every real fine-tuning project this year, each teaching something different and needing a different kind of data.
| Method | What it teaches | Data you need |
|---|---|---|
| Supervised fine-tuning (SFT) | Direct behavior cloning: given this input, produce this output | Input/output pairs, curated or hand-labeled |
| Preference tuning (DPO) | Ranking: prefer this response over that one, without a single perfect target | Response pairs with a chosen-vs-rejected label |
| Reinforcement fine-tuning (RFT) | Reward-driven reasoning, for tasks with a checkable correct answer | A verifiable reward function, not just labeled examples |
Supervised fine-tuning is the default starting point and the method the walkthrough below assumes. It is the right choice whenever you can write down the correct answer for a representative sample of your real traffic.
Preference tuning skips the need for a single perfect answer, useful when quality is a matter of degree or tone and your reviewers can rank two outputs faster than they can write a gold-standard one from scratch.
Reinforcement fine-tuning is the newest and narrowest, limited to reasoning models on tasks with a checkable correct answer: code that passes tests, a math result that checks out. OpenAI lists RFT among its supported methods for reasoning models like o4-mini, alongside SFT, DPO, and vision fine-tuning (OpenAI, Model Optimization guide). No automatic correctness check means no RFT; use SFT or DPO instead.
Full fine-tuning vs. PEFT: why LoRA and QLoRA won
Full fine-tuning updates every weight, producing a complete new copy of the model per task. Parameter-efficient fine-tuning (PEFT), chiefly LoRA and QLoRA, trains a small set of additional weights and leaves the base model frozen. In 2026, PEFT is not the budget option; it is the default, and full fine-tuning is the exception reserved for teams with a specific reason to want it.
The numbers explain why. LoRA's original paper reports cutting trainable parameters 10,000 times and GPU memory 3 times versus full fine-tuning of GPT-3 175B with Adam, while matching or beating full fine-tuning quality on RoBERTa, DeBERTa, GPT-2, and GPT-3, with no added inference latency (Hu et al., LoRA, arXiv:2106.09685). QLoRA pushed further by adding quantization: it fine-tunes a 65-billion-parameter model on a single 48GB GPU while preserving full 16-bit fine-tuning performance. The resulting Guanaco model trained in 24 hours on one GPU and reached 99.3% of ChatGPT's quality on the Vicuna benchmark (Dettmers et al., QLoRA, arXiv:2305.14314).
There is a second, quieter reason PEFT won: storage. Hugging Face notes a 40GB model like bigscience/mt0-xxl produces a 40GB checkpoint per task under full fine-tuning, versus a few megabytes per task with a PEFT adapter (Hugging Face, PEFT blog). That is why teams keep one base model in memory and swap small adapters per task or customer, instead of hosting a full copy for every fine-tune they have ever shipped.
Building a dataset you can trust
The method matters less than the dataset. A LoRA run on five hundred clean, representative examples beats a full fine-tune on fifty thousand noisy ones, almost every time. Three rules hold across SFT, DPO, and RFT.
- Quality over volume. Every example should look like real production traffic, not a cleaned-up version of it. Easier training data than what the model sees in the wild makes the fine-tune brittle exactly where it matters.
- Label with discipline. Write a rubric before you write examples. Two reviewers labeling the same input differently means the model learns the average of their disagreement, worse than either answer.
- Hold out a set you never train on. Reserve 10 to 20% of your data, stratified across categories that matter. It is the only honest way to measure whether the fine-tune generalizes or just memorized.
Choosing where to run the job: hosted API or open-weight and your own compute
This used to be a simple trade-off: hosted fine-tuning for convenience, open-weight for control. In 2026 it is no longer simple, because one side is shrinking. OpenAI is winding down its fine-tuning platform: closed to new users, and existing users can still create training jobs "for the coming months," with models you have already fine-tuned staying available for inference only until their base model is deprecated (OpenAI, Model Optimization guide).
Build around a provider's SFT endpoint and you are not buying a capability, you are renting one on a roadmap you do not control. The durable default in 2026 is an open-weight model, Llama, Qwen, Mistral, or similar, fine-tuned with LoRA or QLoRA on compute you own or rent by the hour. You keep the adapter. You keep the base model. Nobody sunsets your access to either.
Hosted platforms still make sense when you need a fine-tune fast and accept re-evaluating the dependency every year. For anything running past a few months, budget the setup time for open-weight plus PEFT instead. I go deeper on when the fine-tune is worth the maintenance at all, and when it is not, in Fine-Tuning or Not.
Step-by-step: a LoRA/QLoRA fine-tune walkthrough
Here is the shape of a real LoRA or QLoRA run, illustrative rather than a literal transcript, close to a first fine-tune on an open-weight model.
1. Pick the base model, sized to your budget. An 8B to 14B model handles most narrow tasks.
2. Load it quantized. QLoRA loads the base model in 4-bit precision, letting a large model fit on a single consumer or mid-range GPU, then attaches a small LoRA adapter on top.
3. Train, held-out split reserved. Run two to four epochs and watch the held-out loss, not just the training loss.
4. Merge or keep the adapter separate. Separate is usually the better call, swap it at inference time when serving multiple fine-tunes off one base. It is what makes the PEFT storage math hold up in production.
5. Evaluate before you ship. Run the held-out set, and also your full general-capability eval suite, not just the narrow task you tuned for. This is the step teams skip, and the one that catches the failure mode below.
Evaluating the result and naming the failure modes
A fine-tune that nails your held-out eval set can still be a worse model. Two failure modes explain most of the gap, and neither shows up unless you go looking for it.
Catastrophic forgetting. The model reshapes toward your training distribution and quietly loses capability it never got retrained on. A model fine-tuned hard on support-ticket tone can degrade at basic formatting or arithmetic it used to handle fine, because nothing protected those capabilities during training.
Overfitting to training-set phrasing. The model learns the narrow shape of your examples rather than the underlying task. It performs well on inputs that look like training data and fails on real phrasing worded differently, exactly the gap a held-out set too similar to the training set will not catch.
Picture a team that fine-tunes a model to nail a rubric-scored eval for customer tone, ships it, and two weeks later discovers it has started mangling order numbers the base model never touched. Nobody tested order-number formatting, because the fine-tune was about tone. That is catastrophic forgetting: a passed eval set is not the same claim as a model that is ready.
The fix is not more training. It is a wider eval, run before you ship: score the fine-tune against your held-out task set and a general-capability regression suite covering what the model used to do well. Regress anywhere in that second suite, and that is your signal to kill it, retrain with a broader dataset, or fall back to prompting the base model for that slice of traffic.
Is fine-tuning still worth it in 2026, or should I just use RAG?
Both, depending on the problem. Fine-tuning is worth it when you need consistent behavior, tone, or output format a prompt cannot reliably hold. RAG is worth it when the model needs facts that change. Most 2026 production systems that need both run a fine-tuned model on top of a retrieval pipeline, rather than picking one and hoping it covers the other's job.
How much data do I actually need to fine-tune an LLM?
Less than most teams assume. A few hundred to a few thousand clean, representative examples, with a held-out set carved out, is enough for most SFT or DPO projects on a narrow task. Volume matters far less than whether the examples look like real production traffic and were labeled with a consistent rubric.
What's the difference between LoRA and QLoRA?
LoRA trains small adapter matrices on top of a frozen base model, cutting trainable parameters and memory versus full fine-tuning without changing precision. QLoRA adds 4-bit quantization to that frozen base model before attaching the same kind of adapter, which is what lets a 65-billion-parameter model fine-tune on a single 48GB GPU while matching full 16-bit performance.
Can I still fine-tune GPT models, or only open-weight models like Llama and Qwen now?
OpenAI's hosted fine-tuning platform is winding down and, as of 2026, closed to new users, though existing users can still create jobs for a limited window and previously fine-tuned models remain available for inference until their base model is deprecated. That shift is a strong argument for building your fine-tuning workflow around an open-weight model you control, using LoRA or QLoRA, rather than a hosted endpoint you do not.
If you are past "should we fine-tune" and need a team that has shipped LoRA and QLoRA runs into production, with an eval suite that catches forgetting before your customers do, that is the kind of build hiring a ViitorCloud ML developer is for. The method is not the hard part. Proving the fine-tune is better everywhere your users touch it, is.