In March 2025 a group led by Mert Cemri read 1,600 failure traces. Multi-agent systems, seven frameworks, annotated by hand.

They were not looking for what these systems can do. They were looking for how they break.

They found 14 distinct failure modes. Then they sorted them into three categories.

System design issues. Inter-agent misalignment. Task verification.

Look at that list and notice what is not on it. Not one of the three is model capability.

Nobody's agent fell over because the underlying model could not reason well enough. They fell over because the work was badly divided, the parties disagreed about what was happening, or nothing checked the answer.

The three categories are an org chart

Translate them out of research language and they become familiar immediately.

System design issues are specification failures. The work was not clearly divided, the roles overlapped or left gaps, somebody's job was never defined.

Inter-agent misalignment is coordination failure. Two parties held different assumptions, information did not get passed, one waited for the other, context was lost at a handoff.

Task verification is checking failure. Nobody confirmed that what came back was right before it was used downstream.

Those are the three ways every human team has ever failed.

Unclear scope. Bad handoffs. Nobody checking. It is what a project manager exists to prevent, and it is what a project manager fails to prevent. Building a multi-agent system means recreating an organization. Organizations do not fail because the individuals are not clever enough.

The taxonomy holds up better than most, incidentally. The authors report inter-annotator agreement of 0.88, meaning independent people reading the same trace mostly agreed on what went wrong.

That figure matters more than it looks. A failure taxonomy where readers disagree is one person's opinion with citations. At 0.88 the categories are describing something real in the traces rather than something in the authors.

How much of the job actually gets done

While that was being categorized, another group measured throughput.

TheAgentCompany, published in December 2024 by Frank Xu and colleagues, builds a self-contained environment that mimics a small software company. Internal websites, internal data, real tasks of the kind somebody would actually be assigned.

The most competitive agent completed 30 percent of tasks autonomously.

That number is worth holding without spin in either direction. Thirty percent of a knowledge worker's task list is a serious result and nowhere close to a replacement. It is also the ceiling in a controlled environment built by researchers. That is a kinder place than a real company. No 11 years of undocumented exceptions. No system that only Priya knows how to reset.

Once is not the measure

The third finding is the one that changes how you should read the second.

τ-bench, from Shunyu Yao and colleagues in June 2024, tests agents in conversation with a simulated user while following domain policies. Their headline single-trial result was that leading function-calling agents scored under 50 percent.

Then they did something the field had mostly skipped. They introduced a metric called pass^k, which asks whether the agent succeeds on the same task every time across k attempts. In the retail domain, pass^8 came in under 25 percent.

So an agent that succeeds roughly half the time on one try succeeds on all eight attempts less than a quarter of the time. Same task, same agent, repeated.

For a demo, one success is the whole story. For a process, the eighth run is the story, because a process runs continuously and nobody watches every instance.

Notice how the two studies interact. Thirty percent of tasks completed autonomously sounds like a starting point you can build on. Under 25 percent success across eight identical attempts means that starting point moves under you. You are not automating 30 percent of the work. You are automating a share of the work that varies run to run, and the variance is not in your reporting.

Adding agents adds interfaces, not just capacity

Put the three findings together and the architecture implication is uncomfortable for the direction the industry is moving.

The instinct when one agent underperforms is to add more agents. A planner, a critic, a researcher, a tool user. It feels like adding staff.

But every agent you add creates handoffs, and the failure taxonomy says handoffs are where the failures cluster. Two agents have one interface. Five have ten. Each interface is a place for context to be dropped, an assumption to diverge, or a result to be accepted without checking.

That is why adding a reviewer agent so often fails to help. You have not added verification. You have added a party who might also be misaligned, and now the specification can be wrong in two places.

Capacity scales with boxes. Failure scales with arrows.

Why a better model does not fix it

This is the part worth taking to anyone waiting for the next release.

If the failures were capability failures, waiting would be a strategy. Models improve on capability quickly and reliably.

Specification, coordination and verification are not capability. A more intelligent agent following an ambiguous specification produces a more confident wrong answer. A better model at each end of a handoff does not create a contract between them. And nothing about model quality supplies a check that somebody has to decide to build.

These are engineering and management problems wearing an AI costume. They get solved the way they were always solved, which is why the list below looks like it came from 1985.

That is good news, incidentally. It means the answer exists.

What actually helps

Narrow each agent until its job can be stated in one sentence, and give it the smallest set of tools that sentence requires. If the sentence needs an "and", you have two agents.

Make the handoff explicit. A structured payload with defined fields, not a paragraph of natural language passed to the next agent to interpret. Natural language between agents is where misalignment lives, because it permits two readings.

This is the one recommendation people resist, because passing prose between agents is what makes the architecture feel intelligent. It is also what makes it unauditable. A field either has a value or it does not. A paragraph can be understood two ways by two parties and nobody finds out until the total is wrong.

Verify deterministically at the join. Not another agent's opinion. A rule that checks the totals, the required fields and the allowed values, and refuses to pass anything that fails.

The word refuses is the important one. A check that logs a warning and continues is not verification. It is a record of the moment you decided not to stop.

And measure with pass^k rather than pass. Run the same task ten times before you believe anything. If it works nine times out of ten, you have found a system that fails weekly in production, and nobody will know which run it was.

That reframing costs nothing and changes what gets approved. The same pilot, measured once, reads as a success. Measured ten times, it reads as a specification for the controls you still need to build.

The test to run this week

Take one multi-agent workflow you are considering or already running. Draw it as boxes and arrows.

Count the arrows, not the boxes. Then for each arrow, answer two questions. What exactly is passed across it, and what checks it on arrival?

Any arrow where the answer to the second question is that the next agent reads it is an unverified handoff. The research says that is where your failures will come from.

Then run the whole thing ten times on the same input. Not once. If the outcomes differ, you have learned the most useful thing available, which is how often this will surprise you after somebody stops watching.

Sources