What Every Multi-Agent System Gets Wrong
Most multi-agent frameworks hide complexity behind abstractions that collapse under production load. Here is what they get wrong, and a principled path out.
Every few months a new multi-agent framework appears promising to make orchestrating LLM agents simple. They all share the same approach: abstract away the hard parts behind a graph runtime, a magic decorator, or an auto-routing loop. Most teams adopt one of these frameworks early, move fast, then hit a wall when they need to debug a production incident or explain cost attribution to their finance team.
These frameworks are well built. They are just built for the prototype, not for the day something breaks in production and you have to explain why.
Hidden state machines
The most common failure mode is the hidden state machine. Frameworks that route tasks automatically must maintain state about what each agent is doing, what has been retried, and what is waiting for a dependency. That state lives inside the framework. When something goes wrong - and in production, something always goes wrong - your only debugging tool is the framework's own log format, which was designed for framework developers, not for you.
Explicit state costs a little typing and buys you every operational answer. When each task transition is a named enum value in a store you control, you can answer the awkward questions with a SQL query.
Implicit contracts at boundaries
Agent-to-agent communication in most frameworks is string-based or dictionary-based. One agent produces a result, the next agent receives it and parses out what it needs. This works in demos. In production, LLM output is non-deterministic and schema drift is a real failure mode. An agent that expects a JSON object with a summary key will break silently if the upstream agent returns a different structure.
Validate with Pydantic at every boundary and the failure cannot happen. It costs microseconds per call.
No cost attribution
Token costs in a multi-agent system accumulate quickly and unpredictably. A single user request might trigger four agent calls, each with its own prompt and completion tokens, each using different models at different price points. Without per-task token accounting, you cannot answer the basic question: what did this workflow cost?
Cost attribution requires tracing from the first request through every agent call. OpenTelemetry provides the infrastructure; the discipline is threading trace IDs through every message, every agent invocation, and every LLM call.
What a principled starting point looks like
These patterns shaped SMELT. Task state is an explicit Pydantic model, every agent boundary is schema-checked, usage is attributed per task, and OpenTelemetry spans run from ingress to completion. There is no runtime graph and no hidden routing.