vvek

What Anthropic learned building a multi-agent research system, and when more agents are worth the cost

A lead agent, parallel sub-agents and a token bill. Lessons from Anthropic's engineering write-up on when multi-agent designs help and when they do not.

Anthropic's engineering team published a detailed account of how they built the multi-agent Research feature in Claude. It is one of the more candid write-ups on the subject, with real numbers and a list of what went wrong. This article pulls out the lessons that apply to anyone designing agents, whatever model or framework you use.

All figures below are Anthropic's own, from their internal evaluations. They describe one system on one kind of task, so treat them as a guide to the shape of the trade-offs and not as a promise.

The design in one picture

The system follows an orchestrator-worker pattern. A lead agent reads the question, makes a plan, and starts several sub-agents that search in parallel. Each sub-agent has its own context window, runs its own searches and sends back a condensed summary. The lead combines the results and decides whether another round is needed. A separate agent then attaches citations to the claims.

flowchart TD
    Q[User question] --> L[Lead agent: plan and delegate]
    L --> S1[Sub-agent 1]
    L --> S2[Sub-agent 2]
    L --> S3[Sub-agent 3]
    S1 --> L
    S2 --> L
    S3 --> L
    L --> C[Citation agent]
    C --> R[Final report]

This differs from classic retrieval, where a fixed set of chunks is fetched once. Here the search is adaptive: what the agents find changes what they look for next.

What the numbers say

  • A setup with a stronger model as lead and smaller models as sub-agents beat a single stronger agent by 90.2% on their internal research evaluation.
  • On a browsing benchmark they analysed, token usage alone explained 80% of the variation in performance. Adding the number of tool calls and the choice of model brought that to 95%.
  • Agents use about 4 times the tokens of a normal chat. Multi-agent systems use about 15 times.
  • Running work in parallel cut research time by up to 90% on complex queries.

Read these together. More agents help largely because they spend more tokens on the problem, in parallel, each with a fresh context. That is a real benefit, and it is also a real bill.

When it is worth it

Anthropic's own guidance is fairly clear. A multi-agent design makes sense when:

  1. The task is valuable enough to pay for the extra tokens.
  2. The work parallelises well, as broad research does.
  3. The information is more than one context window can hold.
  4. There are many complex tools to use.

It is a weaker fit when agents need to share the same context, when steps depend heavily on each other, or when there is little to run in parallel. They give most coding tasks as an example. If you are choosing between one agent and several, the article on choosing between a workflow and an agent is a good companion: start with the simplest design that does the job.

Lessons that apply to any agent

Brief sub-agents like a new colleague

Early versions gave sub-agents short instructions like "research the semiconductor shortage". The result was duplicated work and gaps. Good briefs contain an objective, the expected output format, guidance on tools and sources, and clear task boundaries. If you would not hand a human contractor that one-line message, do not hand it to a sub-agent.

Scale effort to the question

Models are poor judges of how much effort a task deserves, so put the rules in the prompt. The write-up suggests roughly:

Kind of task Rough effort
Simple fact-finding 1 agent, 3 to 10 tool calls
Comparisons 2 to 4 sub-agents, 10 to 15 calls each
Complex research More than 10 sub-agents with clear divisions

One failure they hit was spawning dozens of sub-agents for a simple question.

Tool descriptions matter more than they look

Each tool needs a distinct purpose and a clear description, because the agent decides which one to use from the description alone. The team went further: they had a model read failed runs and rewrite flawed tool descriptions, and that cut task completion time by 40% for later runs. The same theme appears in designing tools an agent can use well.

Start broad, then narrow

Agents tend to start with long, very specific queries that return little. Prompting them to begin with short, broad searches and then narrow down worked better, much as a person would.

Run things in parallel, in two places

The lead starts three to five sub-agents at once, and each sub-agent calls three or more tools at once. Together this is where the large time savings come from.

Failure modes worth knowing

The list in the write-up reads like a checklist for your own system:

  • Too many sub-agents for a simple task.
  • Searching forever for sources that do not exist.
  • Constant status messages that distract other agents.
  • Vague delegation causing duplicate searches.
  • Choosing search-engine-optimised pages over authoritative sources.
  • Using the wrong tool, such as web search for data that lives in a chat tool.
  • Small errors compounding over long, stateful runs.
  • Non-deterministic behaviour that is hard to debug without full tracing.
  • Updates to the system disrupting runs already in progress.
  • Waiting on sub-agents one at a time, which creates bottlenecks.

Note

The tracing point deserves attention. Agents can make different choices on identical input, so you need to see the decisions they made, not only the final answer. Anthropic describes monitoring agent decision patterns and interactions without reading the conversation content, to protect privacy.

Evaluation: start small

You do not need a huge test set to begin. The team started with about 20 queries that reflected real use, and changes to prompts often showed clear effects on such a small set. Later they added automated grading and human review for things automation missed. If you want a plan for building your first set, see evals for AI agents.

Keeping each agent's context small

A useful side effect of sub-agents is that each works in its own context window and returns only a summary. This is one of the practical ways to keep context focused, which the article on context engineering for agents discusses further. A lead that saves its plan to external memory before the context fills up can also carry on cleanly after a long run.

What to take away

  • More agents mean more tokens. Make sure the task justifies the cost.
  • Write delegation briefs as carefully as you would for a person.
  • Put effort-scaling rules in the prompt.
  • Treat tool descriptions as part of the product.
  • Trace decisions from day one.
  • Evaluate early, even with twenty examples.

Multi-agent designs are a strong tool for broad, parallel work. They are not a default. Start with one agent, measure where it struggles, and add structure when the data tells you to.

Sources

Keep reading

A short email when we publish. Confirm once, unsubscribe any time.