Every AI vendor deck this year has a slide with a dozen little robot icons connected by arrows. Somebody says "agent swarm." Everybody nods.
I want to be careful here, because there is something real underneath the slide, and it is not the swarm part.
One agent with a huge prompt hits a wall
The obvious way to build an AI assistant for a business is to write one enormous system prompt. Everything the model needs to know about your operation — pricing rules, escalation policy, the fourteen exceptions your dispatcher carries in her head — all of it in one block of text, in front of one model, on every single request.
This works fine right up until it doesn't. What happens as that prompt grows is not a crash. It's a slow blurring. The model starts averaging across everything you told it. Instructions that mattered a lot in one narrow situation get diluted by instructions that matter in a hundred others. You add a rule to fix one failure and quietly break two behaviors you weren't testing.
The fix is not a smarter model. The fix is giving each decision a smaller world to reason about.
What routing actually does
Here is the pattern that works, stripped of the robot icons.
One agent reads the request and decides what kind of problem it is. Then it hands a narrow, specific question to one or more specialists — each of which has its own prompt, its own domain, and none of the other twenty-four domains cluttering its context. Each specialist answers the question it was actually asked. The first agent takes those answers back and produces one response.
Three things make that better than the giant prompt:
Context isolation. The security specialist is not reasoning around a paragraph about invoice formatting. It sees a security question and security instructions. Narrower context, sharper answer — this is the whole trick, and it's most of the value.
Parallelism. The specialists don't wait on each other. Three of them working at once costs you roughly the latency of one.
A single owner of the output. This is the part swarm diagrams usually get wrong. If five agents all talk to each other and to the user, nobody owns the answer, and the user gets five overlapping opinions with no recommendation. One agent has to synthesize and commit.
Where swarms actually go wrong
The failure modes are boring and predictable, which is good news, because it means you can design around them.
Unbounded fan-out is the big one. If the routing agent can consult as many specialists as it likes, it will consult a lot of them, because consulting looks like thoroughness. Each one is a full model call. Cost and latency climb, and the marginal specialist adds nothing but a paragraph that agrees with the previous paragraph. You need a hard cap, enforced in code — not a polite request in the prompt.
The second one is ceremony. An agent team answering "what time do you close?" by convening a panel is worse than a single sentence. The router has to be allowed — explicitly, in writing — to just answer.
The third is agents talking to agents in loops. Specialists that can call other specialists produce systems nobody can debug or price. Keep the graph shallow: one lead, one round of specialists, synthesize, done. Add a second round only when the first genuinely didn't settle it.
The example we built
Rather than argue about this abstractly, we built one and left it running.
It's called The Agent Army. A team lead sits in front of a bench of twenty-five specialists — software architecture, cloud, DevOps, SRE, data engineering, ML, security, compliance, mobile, QA, product, and so on. You ask a question. The lead decides whether the bench improves the answer, picks between one and five specialists, runs them in parallel, and comes back with one synthesized response — sections per specialist, then a summary that actually commits to a recommendation.
Every constraint above is in there, because we hit every one of them while building it:
- The five-specialist cap is enforced by the orchestrator. The sixth call is rejected. The prompt asks nicely; the code makes it true.
- The lead is explicitly permitted to answer directly, and it does, often. Greetings and definitions don't get a panel.
- Specialists can't call other specialists. One round, then synthesis.
- The shared system prompt sits behind a cache breakpoint, so the expensive part of the context is billed at a fraction after the first call.
You can go push on it yourself: try The Agent Army. It sits behind a light gate — ask for access and you'll get it.
Two honest notes. It's a demonstration of the pattern, not a product we sell — the point is to show the architecture working on real questions rather than on a slide. And it's tuned for technology consulting questions, because that's the bench we staffed. Ask it something outside that and you'll see the seams, which is itself informative.
What this means if you run a business
Almost nobody needs twenty-five specialists. That number is right for a consulting bench and wrong for nearly everything else.
The version that pays for itself in a regional operating business is usually three or four agents wired into a workflow you already run. Something reads the inbound request and classifies it. Something pulls the relevant history. Something drafts the response. Something checks it against your actual rules before a human sees it. That's an agent team. It has a router, it has isolated contexts, it has one owner of the output, and it has a hard cap — it just doesn't look impressive on a slide.
The architecture is the same. The scale isn't. When somebody shows you a swarm, the useful question isn't how many agents there are. It's: who owns the answer, what's the cap, and what happens when one of them is wrong.
That last one is the real work, and it's a different article.