What We Plan to Measure Before Calling an Agent Production-Ready

Latency and accuracy are the metrics that demo well, which is exactly why they are insufficient as a release gate. An agent can be fast and frequently correct while still being unsafe to deploy, because the failure modes that matter in production are the ones that do not show up in a happy-path benchmark.

The Evaluation Mesh is our planned answer: the design turns AI quality into a release gate instead of an after-the-fact dashboard. It is not built yet. The bar we are building toward is that, before we call an agent production-ready, it has to clear regression scenarios across dimensions that a single accuracy score hides.

The dimensions we plan to gate on

The design is that each of these runs as a repeatable scenario suite, not a one-time audit, and that a change to a prompt, a tool permission, or a retrieval index re-runs the full mesh.

• Tool-boundary violations — did the agent attempt an action outside its granted permissions, even if the attempt was blocked?
• Source coverage — what fraction of factual claims trace back to a retrieved record versus model memory?
• Rollback paths — for every write action, can we reconstruct and reverse it from trace artifacts alone?
• Abstention quality — when the agent should have said "I don't know" or escalated, did it? Confident wrong answers are scored worse than honest abstentions.
• Prompt-injection resilience — does adversarial content in retrieved documents or user input change the agent's permissions or goals?
• Human-review burden — how many runs require an operator, and is that number trending toward fatigue?

Abstention is a feature, not a failure

Most evaluation harnesses penalize a model for not answering. Our design inverts that for high-stakes workflows. An agent that abstains and escalates when retrieval confidence is low is behaving correctly; an agent that fabricates a plausible answer under the same conditions should fail the gate, regardless of how often it is right elsewhere.

This is why the persona split matters operationally. Abbey's analytical, safety-oriented review is meant to flag a low-confidence retrieval before Abi's action-oriented execution profile ever touches production data.

Review burden as a leading indicator

The metric teams most often forget is the one that predicts whether their controls will survive contact with reality: human-review burden. If every run needs a human, the system does not scale. If no run needs a human, the gates are probably theater. The plan is to track the ratio over time and treat a sharp move in either direction as a signal that the control plane needs retuning, not just the model.