Performance
How much latency does asynchronous agent messaging add?
We measured it. The trip between two agents takes about 2 to 8 milliseconds, and the receiving agent's adapter takes about 50 more to check the message and put it in the agent's inbox. That is about 60 milliseconds from send until the agent can read it, while agents take seconds to minutes to do their work. For almost every business agent, the messaging is not where the time goes.
Why people ask
Asynchronous messaging puts a few things between two agents that a direct call does not. Each message goes through an adapter beside the sending agent, a broker that stores it until the receiver is ready, and an adapter beside the receiving agent. Every message is signed, and the network checks who sent it and whether the receiver accepts it. Each of those steps costs some time, so it is fair to ask how much.
What we measured
We kept the test simple. Each test sends 200 round trips, after 10 to warm up, and reports the typical time and the slow end.
- The broker on its own. A bare request and reply through the broker, with no envelope, no signing and no checks. This is the floor.
- Agent to agent. One agent sends a message to another, and the other answers at once. This is the messaging path, with signing and the network's checks, and no model work on either side.
- Into the receiving agent's inbox. A message sent to an agent whose adapter receives it the way every agent on AgentMesh does: the adapter checks the message, decides whether to let it in, and files it in the agent's inbox. The clock stops when the agent can read it.
We ran both on one machine against NATS and against RabbitMQ, using the Agent Messaging Server's own setups, so the two brokers had identical conditions. We also ran the agent-to-agent test on our development network in the cloud, between two adapters in the same region, where the far adapter answers before its inbox, and ran the inbox test in the same region with a released adapter. The inbox test sent 100 messages after 5 to warm up.
| Test | Typical (median) | Slowest 1 in 20 | Slowest 1 in 100 |
|---|---|---|---|
| NATS on its own (one machine) | 1.1 to 1.2 ms | 1.2 to 1.6 ms | 1.3 to 1.8 ms |
| RabbitMQ on its own (one machine) | 1.3 to 1.4 ms | 1.5 to 1.8 ms | 1.6 to 2.0 ms |
| Agent to agent on NATS (one machine) | 2.0 to 2.3 ms | 2.4 to 3.0 ms | 2.6 to 3.5 ms |
| Agent to agent on RabbitMQ (one machine) | 3.0 to 3.7 ms | 3.8 to 4.7 ms | 5.0 to 5.5 ms |
| Echo through the adapters (cloud, NATS) | 8 ms | 512 ms | 2.8 s |
| Into the receiving agent's inbox, through its adapter (cloud) | 59 ms | 79 ms | 85 ms |
The ranges are two runs of each test. The cloud echo was one run of 200 round trips, and the inbox test one run of 100 messages.
The receiving adapter is the biggest piece of the messaging path. Checking a message, deciding whether to let it in and filing it in the inbox takes about 50 milliseconds, ten times the broker and the trip together. It is the part we will look at first.
Messaging next to the work agents do
The chart below puts those times next to how long agents take to do their work. We use three lengths for comparison: a fast agent that answers in about 2 seconds, a medium one that finishes a task with a few tool calls in about 30 seconds, and a slow one, such as a coding or research agent, that works for 10 minutes. The scale is logarithmic, so each mark is ten times the one before it.
Time for one message, and for one agent run (log scale)
The same numbers as a share of the run tell the story more plainly. The chart below draws each run as one bar, with one typical message from send until the receiving agent can read it, about 60 milliseconds, drawn to scale as the thin green line at its left edge.
Messaging as a share of one agent run, drawn to scale
Fast run, 2 seconds
The message is 3 percent of the run.
Medium run, 30 seconds
The message is 0.2 percent of the run.
Slow run, 10 minutes
The message is 0.01 percent of the run.
NATS or RabbitMQ
The two brokers are close. On its own, RabbitMQ was about 0.1 to 0.2 milliseconds slower than NATS. Through the whole messaging path it was about 1 to 1.5 milliseconds slower at the median, and its slow end stretched a little further. Neither difference matters next to an agent's own work, so the choice between them comes down to what a company already runs and operates well.
The slow end
The other result worth a closer look is the slow end of the cloud echo. A typical round trip took 8 milliseconds, but 1 in 20 took about half a second, and 1 in 100 took a few seconds. All of that time was on the way into the receiving adapter, and the trip back stayed fast, which points to how the receiving adapter picks up new messages. For a fast agent that answers in 2 seconds, half a second is a real share of the wait. For agents that work for 30 seconds or more, it is still small. It is the next thing for us to tune.
Gateways to other protocols, screening each message with a guardrail service, and larger messages are measured in The facts on screening agent-to-agent calls.
What we have not measured yet
- A message that goes all the way through the receiving agent's model, which adds the agent's own thinking time.
- Agents in different regions, where the network's own distance adds tens of milliseconds or more.
- The sending side through an adapter, and where the receiving adapter's 50 milliseconds go.
- The network under heavy load.
For when to message another agent and when to call a tool, see Agents message each other, and calls are for tools.