Orchestrating agents without losing the plot

The first version worked in the demo and drifted in production. Not dramatically — no crashes, no obvious failures. Over three weeks the escalation rate climbed from four percent to nineteen, and nobody noticed because no single trace looked wrong.

The seam, not the model

Each agent in the network did its job. The supervisor routed correctly. What went wrong lived between them: a summarisation step that lost the qualifier “pending counterparty approval”, which the downstream agent then treated as settled.

This is the failure mode nobody warns you about. The models were fine. The interface between the models was lossy, and loss compounds along a chain.

What we changed

Three things, in order of how much they mattered.

We made the handoff schema explicit. Every message between agents now carries a typed structure rather than prose. If a qualifier exists, it has a field. If a field is dropped, the schema fails loudly instead of the meaning quietly evaporating.

We evaluate the seams, not just the endpoints. The eval suite now has cases that assert on intermediate state. An agent that produces the right final answer for the wrong reason fails the suite.

We capped the chain depth. Beyond four hops, our ability to reason about what went wrong collapsed, and so did the client’s. Depth is a cost you pay in debuggability.

What we would tell you before you start

If you cannot draw the state that passes between two agents on a whiteboard, you are not ready to build the third one. The appeal of agent networks is that they seem to absorb ambiguity. They do not. They relocate it into the gaps between components, where it is harder to see and much harder to test.

Start with one agent and a very good eval harness. Add the second only when you can say precisely what it receives.