Multi-Agent Systems: When to Split One Agent Into Many

A multi-agent system coordinates several LLM agents through a defined pattern, and it earns its cost only when the task is genuinely parallel or specialized.

A multi-agent system splits one job across several LLM agents that coordinate through a defined pattern: orchestrator-worker, supervisor-hierarchical, or peer handoff. It earns its cost only when the task decomposes into genuinely parallel or specialized subtasks. Most workloads do not clear that bar. A single well-scoped agent is cheaper, easier to evaluate, and fails in fewer places.

I get some version of "should we go multi-agent" in almost every architecture review I sit in on now. The honest answer is usually no, and the people asking are usually relieved to hear it. Multi-agent is not a maturity level you graduate into. It is a specific trade you make when a specific decomposition exists, and you pay for it whether the decomposition was real or not.

Key takeaways

Multi-agent systems cost more before they deliver more. Anthropic's own production research system used roughly 15x the tokens of a single chat interaction to win.
The decomposition test comes first. If a task cannot be split into independent, parallel, or clearly specialized pieces, one agent beats a fleet every time.
There are three dominant coordination patterns: orchestrator-worker, supervisor-hierarchical, and peer-to-peer handoff. Each fails differently.
Multi-agent failures cluster into named categories, not random bad luck. The MAST taxonomy documents 14 distinct failure modes across 1,600+ production traces.
Token usage alone can explain most of the performance gap. On Anthropic's BrowseComp evaluation, token usage explained 80% of the variance in outcomes.

Read this as the coordination-pattern layer under my broader argument in an honest accounting of what agents can do today: agents earn their keep in a narrow band, and multi-agent adds a second axis of narrowness on top of the first.

What is a multi-agent system

A multi-agent system is an architecture where two or more LLM-driven agents each hold their own context, tools, and objective, and coordinate through a defined communication pattern to complete a task neither could complete alone as efficiently. The coordination pattern, not the agent count, is what defines the architecture. Three agents with no defined handoff protocol are not a multi-agent system. They are three single agents that happen to run near each other.

That definition matters because "multi-agent" gets used loosely for anything with more than one model call in it. A pipeline where agent A always hands off to agent B in a fixed sequence is closer to a workflow: the coordination is scripted, not decided. Real multi-agent architectures involve a coordination decision, made by an agent, about who does what next.

When to split into multiple agents vs. keep one

Before you design a coordination pattern, run the task through three questions. If it fails any one of them, stay with a single agent.

Does the task decompose into genuinely independent subtasks? "Research these five competitors" decomposes. "Write a coherent 3,000-word report" does not, because every paragraph depends on every other paragraph.
Do the subtasks benefit from running in parallel? If subtask B always waits on subtask A's output, you have a sequence, not a parallel workload, and a single agent looping through steps is simpler and cheaper.
Do the subtasks need genuinely different tools, context, or specialization? A code-review agent and a security-scanning agent need different tool access and different context windows. A "draft this email" agent and a "draft that email" agent do not; that is one agent run twice.

I have watched teams answer all three of these "yes" in a planning meeting and "no" once the traces come in. The task looked parallel from the whiteboard. In production, subtasks kept needing each other's intermediate results, and the system spent most of its wall-clock time serialized anyway, paying coordination overhead for a sequence it never escaped.

Multi-agent is not a maturity level you graduate into. It is a specific trade you make when a specific decomposition exists, and you pay for it whether the decomposition was real or not.

That is the call worth an outside opinion before you commit to the build. If you want a second read on whether your decomposition is real, hire an AI engineer who has shipped both patterns to pressure-test it before you write the coordination layer.

The orchestrator-worker pattern

In the orchestrator-worker pattern, a lead agent plans the task, breaks it into subtasks, and spawns worker agents to execute each piece in parallel. The lead agent synthesizes the workers' results into a final answer. This is the pattern behind Anthropic's own production research system: a lead agent (Claude Opus 4) plans and delegates, and parallel subagents (Claude Sonnet 4) explore different angles of a query simultaneously.

The result on Anthropic's internal evaluations was decisive. The multi-agent system outperformed a single Opus 4 agent working alone by 90.2% on their research benchmark. That is a real, well-documented win, and it is the number most people cite when they argue for going multi-agent.

What gets cited less often is the cost of that win. The multi-agent system consumed roughly 15x more tokens than a single chat interaction, and on Anthropic's BrowseComp evaluation, token usage alone explained 80% of the variance in performance. Most of the gain was not clever coordination. It was brute-force parallel search, paid for in tokens.

# Orchestrator-worker, roughly what a trace looks like

LEAD agent=opus task="competitive landscape for X"

plan: spawn 4 workers, one angle each

worker_1 model=sonnet angle="pricing" tokens=28k

worker_2 model=sonnet angle="positioning" tokens=31k

worker_3 model=sonnet angle="reviews" tokens=19k

worker_4 model=sonnet angle="hiring signals" tokens=22k

SYNTHESIZE lead reads all 4, writes final report

total tokens = 100k+ vs ~7k for a single-agent pass

Orchestrator-worker is the right pattern when the subtasks are genuinely independent, you can afford the token bill, and the synthesis step does not itself become a bottleneck. That last condition is the one teams miss. Four workers producing 20,000 tokens each hand the lead agent 80,000 tokens to reconcile into one answer, and that reconciliation is where a lot of orchestrator-worker systems quietly go wrong.

The supervisor / hierarchical pattern

The supervisor pattern looks similar to orchestrator-worker but solves a different problem. A single coordinator agent routes each incoming request to one of several specialist agents, based on what kind of request it is. The specialists do not run in parallel on the same task; they are alternatives, and the supervisor picks one.

Think of a customer-support system with a billing specialist, a technical-troubleshooting specialist, and an account-changes specialist, all sitting behind a supervisor that classifies the incoming ticket and routes it. The supervisor is not decomposing a task into parallel pieces; it is choosing which specialized context and tool set applies.

The contrast with orchestrator-worker is the shape of the fan-out. Orchestrator-worker fans one task out to many workers and fans the results back in. Supervisor-hierarchical fans one request out to exactly one specialist and returns that answer directly. The coordination cost is lower, closer to a routing decision than a full parallel search, which is why supervisor patterns tend to be cheaper and easier to evaluate.

The failure mode here is misrouting. A supervisor that classifies a ticket wrong sends it to a specialist with the wrong tools and the wrong context, and that specialist will often produce a confident, wrong answer instead of flagging that it got the wrong kind of request. Evaluating a supervisor pattern means scoring the routing decision separately from the specialist's output quality, because a good specialist cannot save a bad route.

Peer-to-peer and handoff patterns

In peer-to-peer handoff, agents pass control directly to each other without a central coordinator. Agent A works until it decides agent B should take over, hands off with whatever context B needs, and B continues. This is the "conversable agent" model that Microsoft's AutoGen research popularized in 2023: agents that communicate through natural language and code, across mixed LLM, human, and tool configurations, with no fixed hierarchy dictating who talks to whom.

Handoff patterns suit tasks with a natural conversational shape: a drafting agent handing off to a critiquing agent, a planning agent handing off to an execution agent, a sales-qualification agent handing off to a scheduling agent once a lead clears a threshold. The flexibility is also the risk. With no supervisor enforcing structure, nothing stops an early handoff, or two agents looping a task back and forth without progress.

I use handoff sparingly, and only when the handoff condition is explicit and checkable, not left to the sending agent's judgment about when it feels done. "Hand off once the draft passes the schema check" is a handoff condition you can test. "Hand off when you think you're finished" is not, and it is how you get agents that hand off too early or never hand off at all.

The coordination tax: tokens, latency, and cost

Every multi-agent pattern pays a coordination tax on top of the model calls that would exist in a single-agent version of the same task. Three costs stack:

Token cost. Every worker or specialist re-reads relevant context that a single agent would have held once. Anthropic's ~15x figure is the clearest public data point on how large this multiplier can get.
Latency, unless parallelism is real. Orchestrator-worker only saves wall-clock time if the workers genuinely run concurrently. A system that spawns four workers but has them wait on a shared resource pays the token cost of four agents and the latency cost of one slow sequence.
Synthesis and evaluation cost. Someone, or something, has to reconcile multiple agents' outputs into one answer and verify the reconciliation was correct. That step scales with agent count and is easy to under-provision.

The business version of this math is simple: multi-agent is a bet that the quality gain is worth a token bill that is often an order of magnitude higher. On tasks where that gain is real, as it was on Anthropic's research benchmark, the bet clears. On the tasks where a single well-scoped agent would have produced comparable output, you are paying multi-agent prices for single-agent results, and that gap shows up directly on your inference bill.

Why multi-agent systems fail

The failures are not random. Researchers built the Multi-Agent System Failure Taxonomy (MAST) from more than 1,600 annotated traces across seven popular multi-agent frameworks and found 14 distinct failure modes, sorted into three categories.

System design issues. Vague task boundaries, unclear ownership of a subtask, or a coordination pattern that does not match the task's actual shape.
Inter-agent misalignment. Agents duplicating each other's work, talking past each other, or disagreeing about the state of the task with no mechanism to reconcile.
Task verification failures. No agent, and no supervisor, checks whether the combined output is correct before it ships.

The paper's headline finding is blunt: multi-agent performance gains on popular benchmarks are "often minimal" relative to the complexity added. That lines up with what I see firsthand. The most common failure is not a dramatic crash. It is an orchestrator that spawns far more subagents than a query needed, nothing constrains the worker count, and the token bill triples for no corresponding gain in answer quality.

Multi-agent performance gains on popular benchmarks are often minimal relative to the complexity added. The failures cluster into named categories: vague boundaries, duplicated effort, and nobody checking the combined answer.

The second most common failure is duplicated effort from vague task boundaries: two workers assigned overlapping angles both do the obvious research, and the orchestrator pays for the redundancy without getting two independent perspectives. The fix is not a smarter model. It is a sharper task decomposition, written down before the agents run, not improvised on the fly.

What to evaluate before you ship a multi-agent system

A multi-agent eval has to check more than the final answer, because a correct final answer can hide a broken process that will not survive a harder query next week. Score four things:

What to check	Why it matters
Task decomposition quality	Did the split avoid duplicated or overlapping work between agents?
Parallelism, realized	Did workers run concurrently, or did a dependency force them into a slow sequence anyway?
Token cost vs. single-agent baseline	Is the quality gain worth the multiplier? Run the single-agent version as a control.
Synthesis correctness	Did the coordinating agent's reconciliation of worker outputs introduce errors that no individual worker made?

That last row is the one teams skip most often. Every additional agent in the loop is another place a compounding error can enter, and the synthesis step is where those errors get baked into a confident-sounding final answer. My book Agents That Actually Work goes deeper on building the evaluation harness that catches this before a customer does.

Frequently asked questions

How do I know if I need a multi-agent system instead of one agent?

Run the task through three checks: does it decompose into genuinely independent subtasks, do those subtasks benefit from running in parallel, and do they need meaningfully different tools or context. If the answer to any of the three is no, a single well-scoped agent will be cheaper, easier to evaluate, and no less capable for that task.

What is the orchestrator-worker pattern in multi-agent AI?

A lead agent plans a task, spawns worker agents to execute independent pieces in parallel, and synthesizes their results into one answer. It is the pattern behind Anthropic's production research system, which beat a single agent by 90.2% on internal evals while using roughly 15x more tokens to do it.

Why do multi-agent LLM systems fail?

The MAST taxonomy, built from over 1,600 annotated traces, groups failures into three categories: system design issues like vague task boundaries, inter-agent misalignment like duplicated work, and task verification failures where nobody checks the combined output. The paper's core finding is that performance gains are often minimal relative to the complexity added.

Do multi-agent systems cost more to run than a single agent?

Yes, substantially. Anthropic reports roughly 15x the token usage of a single chat interaction for its multi-agent research system, and token usage alone explained 80% of the performance variance on one of its evaluations. Budget for that multiplier before you commit to the architecture, and measure a single-agent baseline so you can prove the gain is real.

If you are weighing an orchestrator-worker build against a single agent that is already close to good enough, that is the exact decision my delivery teams help clients make. Hire an AI engineer at ViitorCloud to pressure-test the decomposition before you build the coordination layer, not after.