Continual Learning for LLMs, Without the Full Retrain

Continual learning for LLMs means updating a model in increments, not retraining from scratch, and no method fully stops it from forgetting.

Continual learning for LLMs is the practice of updating a model's knowledge or behavior incrementally, through parameter-efficient adapters, replay-based fine-tuning, model merging, or targeted knowledge edits, instead of retraining the full model on the whole dataset every time something changes. In production this is a damage-control discipline, not a solved problem. Every technique here narrows catastrophic forgetting. None of them close it, which is why most teams still lean on retrieval or versioned fine-tunes to cover what continual learning has not fixed.

I get some version of this question every time a team ships a fine-tune and then wants to add a second one on top without redoing the first. The honest answer is rarely the one they were hoping for.

Key takeaways

• Continual learning narrows catastrophic forgetting, it does not end it. LoRA, adapters, and model merging all trade off against interference once you stack enough of them.
• LoRA is the practical default for incremental updates. It can cut trainable parameters by up to 10,000x and GPU memory by roughly 3x versus full fine-tuning, with no added inference latency.
• Rehearsal and elastic weight consolidation fight forgetting directly, by replaying old examples or constraining how far important weights can move, at a real cost in complexity.
• Model merging (TIES, DARE) folds in new skills without a joint retrain, at the cost of interference between the merged tasks.
• No major fine-tuning API supports continual fine-tuning on top of an already fine-tuned model. Most teams route around this with RAG, or by retraining from the same fixed base each cycle.

What continual learning for LLMs actually means

Start with what it is not. A one-time fine-tune trains a model once, on a fixed snapshot of data, and then stays frozen until someone decides to retrain it. RAG never touches the weights at all; it hands the model documents at query time and lets the base model reason over them fresh, every call. Continual learning for LLMs sits between those two: it updates the weights, but incrementally and repeatedly, ideally without needing the full original dataset each time you add something new.

That "ideally" is doing a lot of work. The technique you pick, adapters, replay, merging, or a narrow edit, decides how much of the model's prior competence survives the update. None of them decide that perfectly. The rest of this piece is about what each one actually buys you, and where it quietly costs you back.

Why full retraining doesn't scale for continual learning

Retraining an LLM from scratch every time your product, policy, or catalog changes is not a cadence, it is a budget line most teams cannot sustain. A full pretraining run costs millions of dollars and weeks of cluster time at frontier scale, and even a full fine-tune of a mid-sized model on new data means reloading every parameter, backpropagating through the whole network, and paying for an optimizer state sized to match. Nobody ships a new model version every time a support policy changes.

The workaround is incremental learning: keep the model you have, feed it new data, update only what needs to change. The problem that workaround runs into is catastrophic forgetting. A network trained sequentially on new data tends to overwrite what it learned before, sometimes badly, because gradient descent has no built-in reason to protect old competence while it optimizes for the new task. The most comprehensive survey of the field frames this tension around two axes, vertical continuity (general to specialized) and horizontal continuity (across time and domains), and names catastrophic forgetting as the open problem across every stage: continual pre-training, domain-adaptive pre-training, and continual fine-tuning (Shi et al., 2024).

If you have not nailed down what a single fine-tune actually changes in a model's weights, that is worth doing before you stack a second one on top of it. I cover the mechanics in my full fine-tuning guide; everything below assumes you already know what one training run does.

LoRA and parameter-efficient adapters as the practical default

The most common way teams do continual learning for LLMs today is parameter-efficient fine-tuning, LoRA foremost among the methods. LoRA freezes the pretrained weights and trains a small set of low-rank matrices injected next to specific layers, instead of touching every parameter in the network. The original paper found this cuts trainable parameters by up to 10,000 times and GPU memory by roughly 3 times versus full fine-tuning of GPT-3 175B with Adam, with no added inference latency once the adapter is merged (Hu et al., 2021). I go deeper on the mechanics, and the cases where it loses to a full fine-tune, in LoRA vs full fine-tuning.

Picture a support team, purely as an illustration, that ships a new LoRA adapter for a new product line every quarter. The first two adapters land clean. By the third, responses for the original product line start drifting, format slips, tone shifts, because each new adapter nudges the same small set of layers and the nudges start to interfere. Nothing crashes and nothing throws an error. The eval suite on the older product line just gets quietly worse, and nobody is watching that number because the new adapter's own eval looks fine.

That is the honest shape of PEFT-based continual learning: cheap, fast, and it buys real headroom before interference shows up. It does not buy immunity from it.

Rehearsal and elastic weight consolidation

Two older techniques attack forgetting more directly than PEFT does, by working inside the training loop instead of around it. Rehearsal, also called replay, mixes a sample of old training examples back into every new training batch, so the model keeps seeing evidence of what it already knew while it learns the new task. Elastic weight consolidation (EWC) takes a different angle: it estimates which weights mattered most for prior tasks and penalizes the optimizer for moving them far, through a constraint term added to the loss function.

Both work. Both also cost something the LoRA pitch tends to skip. Rehearsal needs you to keep and manage a replay buffer, which gets expensive and sometimes impossible when the old training data sits behind a privacy agreement you no longer have access to. EWC needs you to estimate parameter importance for every prior task before the next one starts, and that estimate degrades as you stack more tasks, the same interference problem showing up one layer deeper. Neither method scales cleanly past a handful of sequential updates without real engineering investment.

Model merging folds in new skills without a joint retrain

Model merging skips training altogether for the new-skills step. Take two or more fine-tuned checkpoints that share the same base model and combine their weights directly, no gradient descent required. TIES-merging trims redundant parameter updates and resolves sign conflicts between models before averaging. DARE randomly drops and rescales a fraction of each model's delta weights before the merge. Model soups average the weights of multiple fine-tunes trained with different hyperparameters. All three fold a new capability into a model in minutes instead of a training run.

The trade-off shows up exactly where you would expect: interference. Merge a code-focused adapter with a customer-support-tone adapter and you often get a model that is worse at both than either parent, because the merge algorithm has no way to know which weight changes matter to which task. I have watched a merged model pass a smoke test on both source tasks and then fail a harder eval on either one, the same silent-regression pattern PEFT stacking produces, arrived at from a different direction.

Knowledge editing for narrow, surgical updates

Knowledge editing takes the opposite approach from merging: instead of blending whole models, it targets a single fact inside one. Methods like ROME (Rank-One Model Editing) and MEMIT locate the specific weights that encode a fact, such as who holds a given role at a company, and overwrite just that association, leaving the rest of the model's weights untouched. This is the narrowest form of continual learning for LLMs: you are not teaching the model a new skill, you are correcting one wrong or outdated fact.

The catch is scope. Knowledge editing is precise for the fact you targeted and unreliable at scale. Edit a handful of facts and the model usually holds up. Edit hundreds, and the edits start to interact in ways nobody predicted, degrading unrelated knowledge the same way a large fine-tune does, just through a different mechanism. It is a scalpel, not a workflow for keeping a model current.

Continual learning for LLMs: methods at a glance

When RAG beats fine-tuning for updating a model

Retrieval-augmented generation sidesteps the forgetting problem by never touching the weights. Instead of training new information into the model, RAG hands the model relevant documents at query time and lets it reason over content that lives outside its parameters entirely. Update the document store and the next query sees the change: no retraining, no merge, no risk of the update degrading something else the model already knew.

That is why RAG wins for anything that changes on a schedule shorter than your training cadence: prices, policies, inventory, anything with a timestamp. Continual learning techniques are the better fit when what you are updating is not a fact but a behavior, a format, a skill the model needs to apply consistently rather than look up. I lay out the fuller version of that split, and the test I run to tell the two apart, in my decision framework for when to fine-tune.

Why continual fine-tuning still isn't a supported production path

Here is the trade-off worth saying plainly: every technique above narrows the forgetting problem, and none of them close it. Stack enough sequential LoRA updates, or merge enough task-specific adapters, and you get interference between them. The model starts trading old competence for new, and an eval suite is the only thing that will tell you before a customer does.

This isn't hypothetical. OpenAI's own model-optimization documentation lists exactly three fine-tuning methods, supervised fine-tuning, DPO, and reinforcement fine-tuning, and none of them describe a supported path for continual or incremental fine-tuning on top of a model you already fine-tuned (OpenAI, Model Optimization Guide). The major commercial fine-tuning APIs are built for one-shot adaptation from a fixed base checkpoint, not iterative continual learning. Most teams route around this by retraining from that same fixed base each cycle, folding old and new data into one run, or by moving the updating problem out of the model entirely and into retrieval.

There is a research path that treats this differently instead of working around it. Google's Titans architecture pairs attention with a separate, learned long-term memory module that can incorporate new information at inference time, scaling effectively to context windows beyond 2 million tokens, without a weight-updating training run at all (Behrouz, Zhong, and Mirrokni, 2025). It is a genuinely different answer to "continual," worth watching, and it is not what most production teams are running today.

If you are deciding whether any of this is worth building versus retraining from a fixed base on a schedule, that decision belongs next to the fine-tune-or-not call generally. I work through both in Fine-Tune, or Don't.

Can you fine-tune an LLM without losing what it already knows?

Not fully. LoRA and other parameter-efficient methods reduce how much a model forgets by touching fewer weights, and elastic weight consolidation and rehearsal reduce it further by protecting or replaying prior knowledge. None of these guarantee zero forgetting. Every fine-tune, however targeted, trades some of the old competence for the new one.

What's the difference between continual learning and just using RAG?

Continual learning changes the model's weights, incrementally, to update its behavior or embedded knowledge. RAG leaves the weights untouched and hands the model relevant documents at query time instead. RAG is the better tool for facts that change on a schedule; continual learning techniques are the better tool for a skill or behavior the model needs to apply on its own, without a document to reference.

Is LoRA the same thing as continual learning?

No. LoRA is one technique you can use to do continual learning, not the whole discipline. It is the most common practical default because it is cheap and fast, but rehearsal, elastic weight consolidation, model merging, and knowledge editing are all separate continual learning techniques with different trade-offs.

How often should you update or retrain a production LLM?

There is no fixed cadence that fits every system. Update when an eval on your target task shows a real gap, not on a calendar. Before you commit to another incremental update, run your existing evals against tasks outside the new training data too, since that is where forgetting shows up first and quietest.

Stacking updates on a model without a way to catch what they quietly break is how a working system turns into a liability nobody notices until a customer does. If you are building a system that needs to keep learning without losing what it already does well, ViitorCloud's ML engineering team builds the eval harness that catches the regression before it ships, not after.