Prompt Chaining: When to Break One Prompt Into Many

Prompt chaining is the technique of breaking one complex task into a sequence of smaller prompts, where each call's output becomes the next call's input. You trade a single fragile mega-prompt for a pipeline of narrow, inspectable steps. It sounds unglamorous, and it is the discipline that separates a demo from something you can run in production.

I reach for it constantly, and I skip it just as often. The rule I use is blunt. If I already know the right sequence of steps, I chain. If the sequence itself is uncertain, and the system has to figure out what to do next as it goes, that is an agent's job, not a chain's. Confusing the two is the most common mistake I watch teams make.

Key takeaways

If you read nothing else, read these.

• Prompt chaining decomposes one task into sequential calls. Each step's output becomes the next step's input, with a checkpoint at every handoff.
• Anthropic names it as one of five core agentic workflow patterns, alongside routing, parallelization, orchestrator-worker, and evaluator-optimizer, with outline, critique, draft as its own canonical example.
• The reliability gap is not subtle. One case study on automated report generation found a chained pipeline hit a 100% success rate while an optimized single-shot prompt failed half the time.
• Chain when you already know the steps. Reach for an agent when the right next step depends on what the model discovers along the way.
• Every added link multiplies cost and latency, and an unvalidated bad output at step two silently poisons every step after it.

What is prompt chaining?

Prompt chaining is the practice of decomposing a complex language-model task into a sequence of smaller prompts, each one narrow enough to verify on its own, with the output of one call feeding directly into the input of the next. Instead of asking a model to plan, draft, and revise a document inside one instruction, you run three separate calls and pass the result forward at each step. The payoff is inspectability: you can see exactly which step produced a bad output, and fix that step alone instead of rewriting the whole prompt.

Prompt chaining vs. chain-of-thought vs. agents

These three terms get used interchangeably, and that sloppiness costs teams real engineering time. Chain-of-thought prompting happens inside a single call. You ask the model to reason step by step before it answers, and the reasoning and the final answer come back together in one response. Prompt chaining happens across calls. Each step is a separate request, with a separate opportunity to inspect, validate, or reroute before the next one fires.

Agents are a third thing again, and the distinction matters more than either of the first two. A chain runs a sequence a human already designed in advance. An agent decides its own next step at runtime, choosing which tool to call or which prompt to run next based on what it observes. The moment your pipeline needs to decide its own path rather than follow one, you have left chaining territory.

What moves between every step in a chain, and what an agent chooses to keep or discard at every turn, is context. I go deeper on that layer in the pillar piece on context engineering, and in the narrower comparison of context engineering versus prompt engineering. The short version here: a chain is only as good as the context you deliberately forward at each hop. Nothing carries over that you did not explicitly pass along.

When to chain prompts, and when not to

The decision rule is this. If you can write the sequence of steps down in advance, and each step's job stays stable across inputs, chain. If the model has to decide what step comes next based on what it discovers along the way, you need an agent, not a chain.

Chaining also loses to a single well-written prompt more often than the pattern's fans admit. If one prompt reliably does the whole job, a chain adds latency and cost for no benefit. I only reach for a chain once a single call is measurably failing at the combined task, never because chaining is the more sophisticated-looking choice. If the failure is in how the prompt is written rather than in the shape of the task, fix that first. I cover the specific techniques that close that gap in prompt engineering techniques that move accuracy in practice, and I only chain once those are exhausted.

The deeper, systems-level version of this decision, versioning prompts, handling the long tail of inputs, building the observability that turns three chained calls into something you can operate at 3am, is the subject of my book From Prompt to Pipeline. It is the unglamorous middle between a prompt that worked once and a system someone else can run on call.

The pattern: decompose, sequence, pass context, validate

Every chain that holds up in production follows the same four moves, in this order.

• Decompose. Split the task into steps small enough that each one has a single, checkable job. "Summarize, then extract the three action items" is two steps. "Summarize and extract action items and format them nicely" is one step pretending to be simple.
• Sequence. Order the steps so each one only needs what came before it. A critique step needs the draft. It does not need the original research notes unless you deliberately forward them.
• Pass context. Decide explicitly what moves from one call to the next. Default to passing less, not more. Extra context you did not intend to forward is exactly how a chain drifts off task.
• Validate. Check each step's output against a concrete rule, a schema, a length bound, a required field, before it becomes the next step's input. This is the step most teams skip, and it is the one that matters most.

A worked example: outline, critique, draft

Anthropic's own engineering guidance names prompt chaining as one of five core agentic workflow patterns, alongside routing, parallelization, orchestrator-worker, and evaluator-optimizer. Its canonical example is document writing: write an outline, check the outline against a set of criteria, then write the full document from the validated outline. I use a version of this constantly for anything longer than a few paragraphs.

The reason this beats one big "write me a great document" prompt is the checkpoint. A single mega-prompt asks the model to plan, draft, and self-critique in the same breath, with nothing catching a bad plan before it becomes a bad draft. The chain gives you that checkpoint for the cost of two extra calls.

The research backs this up past my own experience shipping it. A controlled study on text summarization found prompt chaining consistently beat a "stepwise" prompt that tries to combine drafting, critique, and refinement into a single instruction. The stepwise version's self-critique step often simulated a revision without materially changing the output. The chained version's separate critique step caught things, because it was a genuinely independent pass rather than the same call reasoning about itself.

The gap widens further in production systems. A 2026 case study on automated scholarly report generation reported a prompt-chained pipeline reaching a 100% success rate on its test set, while an optimized single-shot baseline prompt failed in half of its runs. That is not a marginal improvement. That is the difference between a system you can put in front of a customer and one you cannot.

Tools for building a prompt chain: API calls, LangGraph, or DSPy

You do not need a framework to chain three prompts. A loop that calls the API, checks the response against a rule, and calls it again is a prompt chain. I have shipped chains that are nothing more than a few sequential function calls and a handful of "if this validation fails, retry" branches.

Reach for an orchestration layer once the chain grows branches, needs to persist state between steps, or has to resume from a failure partway through. LangGraph models the chain as an explicit graph with checkpoints, which pays off once you need conditional routing or a human approval step mid-chain. DSPy takes a different angle. Instead of hand-writing each prompt, you define the pipeline's structure and let it optimize the prompts against a metric you supply, which is worth the setup cost when you are running the same chain pattern across many inputs and want the prompts themselves tuned, not just the plumbing around them.

My honest rule: start with raw API calls. Add an orchestration layer only when you can name the specific problem it solves, branching, persistence, or prompt optimization at scale, and not because a framework is trending this quarter. Most three- or four-step chains never outgrow a plain script, and the extra dependency is a cost you pay on every future change to the pipeline.

The honest cost of prompt chaining: error propagation, latency, spend

Chaining buys predictability at a direct price, and I want to name it plainly rather than bury it. Every additional link in the chain is another sequential API call, another few hundred milliseconds, another line on the bill. A three-step chain costs roughly three times what one call costs, and the user waits for all three, in order.

The sharper failure mode is error propagation. A bad output at step two does not announce itself. It flows into step three as if it were correct, and step three builds on it, and step four builds on that, until the final output ships something plausible and wrong several steps deep. I have watched a chain look healthy in every log line and still deliver a broken answer, because nothing checked the middle. Validate at every hop, or you are adding places for a silent error to hide.

There is a business-facing version of this cost too. Every hop you add to a customer-facing chain is latency the customer feels and spend that shows up in your margin. Before adding a fourth step, ask whether the checkpoint it buys is worth the time and the money, or whether you are chaining out of habit. The honest answer is often no.

Frequently asked questions

What is prompt chaining in AI, exactly?

Prompt chaining is breaking one complex task into a sequence of smaller prompts, where each call's output becomes the next call's input. Instead of one prompt trying to plan, draft, and revise at once, you run separate calls for each step and can inspect, validate, or correct the result at every handoff.

How is prompt chaining different from chain-of-thought prompting?

Chain-of-thought prompting happens inside a single call. The model reasons step by step and returns its answer in one response you never see broken apart. Prompt chaining happens across multiple calls, each a separate request you can validate before the next one runs. Chain-of-thought is a prompting technique. Prompt chaining is a pipeline architecture.

When should I use prompt chaining instead of one big prompt?

Chain when you already know the sequence of steps and a single prompt is demonstrably failing at the combined task. If a well-written single prompt reliably does the job, chaining adds latency and cost for no gain. If the right next step depends on what the model discovers as it goes, you need an agent instead of a chain.

Do I need LangGraph or DSPy to build a prompt chain, or can I just call the API directly?

You can build most chains with a plain loop of API calls and a validation check between steps, and that is often the right amount of engineering. Reach for LangGraph when you need explicit state, checkpoints, or conditional branching, and for DSPy when you want the prompts themselves optimized against a metric across many inputs.

Prompt chaining is not the sophisticated choice or the simple one. It is the right choice exactly when you already know the steps and need to catch a bad one before it reaches a customer. Decompose the task, sequence the steps, pass forward only the context each step needs, and validate at every hop. Skip any one of those and you have built a slower, more expensive version of the mega-prompt you were trying to replace.

If you are building a pipeline that has to hold up outside a notebook, with retries, validation, and cost discipline designed in from the first step, that is exactly the work my team takes on. Bring in a team that has shipped multi-step AI pipelines in production before you spend a quarter debugging the handoffs yourself.