Why AI Agents Go Rogue: Common Failure Patterns and How to Remedy Them
When most people picture an AI agent failing, they picture a red alert, a 500 server error, or a massive stack trace. In reality, production failures almost never look like that.
Across the deployments we monitor, the incidents that truly destroy user trust throw no technical errors at all. A query runs and the response comes back seemingly fluent, confident, and well formatted.
It’s also completely wrong, and nothing in the output gives you the signal that you’re looking at a mistake.
An analyst at one of our customer accounts summed up the frustration mid-session after an agent gave them four different numbers for the exact same metric:
“You keep making mistakes… you are driving me mad today… can I trust your data at all? Please start from scratch.”
By that point, the analyst had spent their morning hand-correcting almost every single figure in a weekly report before sending it out. Yet on paper, the agent’s technical error rate was zero percent.
That’s the real failure mode. AI agents don’t usually break by crashing, but rather they fail by drifting quietly while pretending everything is completely fine.
The Failure Patterns Across Production Agents
When you look across different industries and tech stacks, five specific failure patterns keep popping up.
1. Doing Math in Prose
We see this happen all the time. An agent runs a perfectly valid SQL query, fetches correct raw data, and then decides to compute summary totals or growth rates inside its textual answer instead of letting SQL do the work.
We saw this with a volume reporting agent for one of our customers. It reported a week-over-week change of +19.6%. But if you did the math on the two averages it had pulled right off the screen (85,000 and 77,000), the real answer was +10.4%. The query was fine; the agent just tried to do subtraction in prose and flubbed it. LLMs are notoriously bad at arithmetic, but they state a wrong calculated number with the exact same confidence as a right one.
2. Confusing “I Can’t See It” with “It Didn’t Happen”
When an agent can’t access data at the specific detail level a user asks for, an empty result is ambiguous. It could mean zero business activity, or it could mean the data only exists at a higher, aggregated level.
Models almost always narrate this as “zero activity”. In one instance, our customer asked three times for a breakdown before the agent finally admitted the data did exist—it just couldn’t see it at that lower grain. A “zero activity” answer gets repeated downstream as a business fact, whereas a “visibility” answer prompts the user to ask the right follow-up.
3. Silent Substitution
If an agent can’t find the exact target metric or market code you asked for, it will often grab whatever looks closest in its memory without telling you.
We watched an agent generate a report titled for one specific geographic market, while the underlying filter showed it had queried an entirely different region. Its internal reasoning trace literally read: matched to the closest-looking code from memory. When the customer pushed back, it quietly re-ran against a third market, changing the headline numbers completely while leaving both conflicting versions sitting right next to each other in the thread.
4. Internal Contradictions
Agents frequently contradict themselves within a single response—or across a multi-turn chat—and just keep rolling with it.
In one of our customer’s report drafts, an agent put one total in its data table and a completely different total in the written summary directly above it. Because both sections were internally coherent, standard evaluation pipelines passed it with flying colors. The flaw only showed up when a human lined the two numbers up. In another case, an agent corrected itself in a later turn but never acknowledged that the earlier draft was wrong, leaving the analyst to guess which number was real.
5. Moving the Goalposts Every Time You Ask
When agents generate fresh SQL from scratch for every question, they also re-derive their modeling logic every time.
Ask the same weekly sales performance question twice, and you might get two different answers. In one agentic deployment, two identical runs produced totally different figures for a territory simply because the agent chose a static monetary exchange rate on the first run and pre-converted amounts on the second. Neither run stated its assumptions, and the discrepancy completely changed the year-over-year story.
A Four-Part Framework for Rebuilding Trust
If traditional monitoring stays green through all of this, what should you actually track? The reality is that, for true agent trust, you need observability across four distinct layers:
Context: Is the ground underneath it solid?
Your agent is only as reliable as the underlying tables. If an upstream staging table stalls or gets corrupted, your agent will confidently output corrupt insights.
At one global retailer, a table feeding an executive BI agent started throwing off a sudden spike in negative on-hand-unit values, and the table’s row count jumped in two distinct steps over four days. Any answer that agent gave about inventory during that window would have carried the problem straight into an exec conversation. Nobody had ever connected that table to the agent’s reliability.
This is the pillar teams skip most often, and it’s the one that invalidates the other three. Go check your agent’s lineage. We routinely find agents reading from tables that nobody monitors at all—usually views staged specifically for the agent, with the actual source somewhere else entirely. A stalled refresh upstream is an agent problem.
Performance: Is it running well?
Track latency, token consumption, and invisible retry loops. This is the cheapest layer to stand up and it needs almost no configuration—anomaly detection learns your agent’s normal patterns and flags the drift.
One manufacturer’s reporting agent had runs stretching well past two minutes. Trace analysis showed why: the agent was answering iteratively—plan a query, run it, plan the next one—landing between 10 and 19 SQL executions on a typical run. A handful of outliers hit 35 to 48, and those were exactly the runs where users complained the agent was hanging.
Here’s an important factor: high query counts often include first attempts the agent quietly rewrote and retried. Users never see those, they just see the wait, and it’s a direct impact on both latency and token spend.
Decide what response time your users should be able to count on, then measure against it as an SLO.
Behavior: Did it take sensible steps?
Watch what the agent did, not just what it said. Did it query the data at all when it should have? Did it call tools in the right sequence? Did the task actually finish?
We saw a document-generation agent where users asked for a document and the agent simply never saved it—the conversation just quietly ended. And here’s a failure that’s easy to miss entirely: a tool call reported an error while the step itself wasn’t marked as one, so error-rate monitoring stayed green the whole time the feature was broken.
One caution from experience: don’t monitor this too bluntly. An early “did it run a query?” monitor fired on every single turn without SQL—clarifying questions, acknowledgments, follow-ups already answered by data pulled earlier in the thread. All of them legitimate. The fix was to judge whether a query should have run, not whether one did.
Output: Is the answer actually good?
Run evaluations on the response itself: groundedness, helpfulness, task completion, and user frustration. Groundedness asks whether the text actually reconciles with what the queries returned—not just the numbers, but names, IDs, dates, and scope.
Frustration evals are helpful yet often underrated. They catch the sessions where nothing technically breaks but the user walks away without a usable answer.
On one drafting agent, an eval tracked how often users had to correct the output before landing on a final version, which gave the team a direct read on whether their prompt and template changes were actually working. Another scored whether the agent proactively flagged brand and compliance issues—added after a restricted phrase slipped through and was caught only because a user happened to notice.
Catching sessions like these tells you far more than uptime metrics ever will.
Issues, root causes, and fixes
| What you see | Root cause | The fix |
|---|---|---|
| Summary totals don’t match the chart directly above them | The agent computed the rollup in prose instead of in SQL | Define aggregations as measures in the semantic model. Add a response instruction: “Only state numbers returned by a query — never compute totals, averages, growth, or peaks in text.” |
| Agent reports “no activity” for data it simply can’t see | An empty result is ambiguous and the model narrates it as zero | Declare the grain in the semantic model’s own description so it travels with the data. Pair it with a verified query that checks whether the entity exists, so “not in this data” comes from a lookup. |
| Agent answers about the wrong entity, with the right title | It matched the name to the closest-looking code from memory | Extend existing clarification rules to cover market and exchange references. Add search over the dimension so user terms are matched against real values before SQL is generated. |
| Same question, different answer run to run | The agent re-derives modeling choices — currency basis, date window — on every run | Pin recurring definitions as certified metrics or verified queries so every run resolves identically. |
| A recurring report needs manual correction every week | Core figures are regenerated from scratch each time | Pin the report’s core pulls to verified queries, so it always draws from one trusted, reproducible source. |
| Agent hangs on broad questions | It answers query-by-query instead of consolidating — 14 queries for one question | Pre-build the recurring numbers as a metric or view. Add a certified example query so the agent follows that pattern instead of rebuilding it. |
| Users keep asking for data the agent has no way to reach — a longer history, a table that was never loaded | The data doesn’t go back far enough, or isn’t loaded at all | Not a quality issue — a demand signal. Every one of those requests ends in the manual pull the agent was supposed to replace. Treat it as roadmap input. |
| Results captured, but responses missing from your logs | The agent runs under each user’s credentials, so results aren’t readable by anyone else | Enable conversation sharing at the platform level so full conversations, results included, can be observed. |
What Trips Up a Lot of Teams
When teams first catch an agent making arithmetic errors in text, the immediate instinct is to build another LLM evaluator downstream to check the math.
That’s the wrong altitude.
The real goal isn’t monitoring bad math, but rather making the mistake itself structurally impossible. Force calculations into SQL or code execution tools, and enforce strict response rules: “Only state numbers returned by a query—never compute totals, growth, or averages in prose.”
The same applies to consistency: pinning standard definitions as certified metrics inside a semantic model beats trying to catch logic drift after the fact every week.
After all, monitoring shouldn’t be your primary defense against bad outputs. It should simply be the tool you use to confirm that your structural guardrails are holding up.
Examples in this piece are drawn from production agent deployments across many different sectors. All customer names, agent names, product identifiers, and conversation details have been removed or generalized, and figures are illustrative rather than specific.
