Garbage data, garbage agents: Scaling your Cortex Fleet starts here

I had the privilege of speaking at Snowflake Summit last week, and I chose to focus on a critical issue within the data and AI stack that most enterprises are dealing with but haven’t figured out yet: why agents keep failing in production — and why the answer almost always traces back to data.

Everyone’s building agents, yet few are scaling them reliably in production.

That’s not an opinion, it’s what 260 engineering practitioners told us in Monte Carlo’s 2026 Builders Report. 46% of teams have agents in full production, while 64% said they deployed faster than their teams were ready to support. What’s more, 54% expect to significantly rebuild or rearchitect systems they’ve already deployed.

The question isn’t whether enterprises are moving fast on AI. They are. The question is why so many of them keep hitting the same wall.

The answer, more often than not, is data.

The agent hallucinated. Or did it?

When a Cortex agent gives a wrong answer, the instinct is to blame the model. Teams argue t hat they need better prompts, smarter retrieval, or even a different LLM.

But most of the time, what you think is wrong with the agent — “Cortex hallucinated,” “the model isn’t smart enough,” — is masking what’s actually wrong many layers beneath that, such as a Snowflake table that hasn’t refreshed in days, a Cortex Search index that never updated after a schema change, or a Semantic View built on an outdated metric definition.

The agent didn’t hallucinate, it told the truth about the data; it’s the data itself that was bad.

This is the garbage-in multiplier. Data and AI are two halves of a holistic system — data powers agents, and agents produce data that flows back into the system. A problem at the data layer doesn’t stay at the data layer; it travels and compounds. By the time it surfaces as a wrong answer at the agent output, it’s already passed through document processing, embedding models, search indexes, and retrieval — each step amplifying the original failure.

Stale tables. Row counts dropped. Schema changed. Nobody’s watching. That’s where most agent failures actually start.

What’s beneath the surface of the iceberg

Looking again at our recent study, we found that 59% of data and AI practitioners are not fully confident they have visibility into the data and tools their agents access. 49% have discovered that an agent was accessing data or systems they weren’t aware of, and 64% say incorrect or hallucinated responses limit their trust in AI.

The pattern is consistent: teams see the symptom at the surface — the wrong answer, the slow response, the failed recommendation — and miss the root cause underneath. What you don’t see includes a source table not refreshing, a schema change that silently nulled out half the fields, retrieval returning the top result by similarity score rather than recency, an API rate limit causing intermittent failures with no retry logic.

Most agent observability tools stop at the agent layer itself. They tell you something is wrong. They don’t tell you why — and they definitely don’t tell you it started three hops upstream in a Snowflake task that’s been failing for hours.

Four layers. Most teams only watch one.

Scaling a Cortex fleet reliably in production requires observability across the full stack — not just the agent itself.

The data layer is the bedrock: every agent decision is downstream of it, and most teams aren’t monitoring it for AI workloads at all. The semantic layer — RAG, embeddings, vector store, metadata — is where every upstream problem gets amplified. The agent-build layer covers orchestration, tool use, multi-agent behavior, and trajectories. And the trust layer isn’t a box on top — it’s an observability plane running through all three layers: lineage, anomaly detection, impact analysis, auto-resolution.

Most teams are only watching layer three. The agent logs, the traces, and the evals. That’s reactive — and I would argue that it’s not enough.

The teams building for production, not pilots, instrument all four.

What production-grade actually looks like

During my session at Snowflake Summit, I traced a live example to illustrate the data origins of many agentic failures via a flight booking agent giving travelers recommendations on what to book. In this example, the agent’s recommendation validity dropped from 99.2% to 94.6%, generating 34 user complaints in six hours. For most engineers, the instinct would be to look at the model or the prompts.

The root cause, in this case, was a data failure: the flights_inventory Snowflake table hadn’t refreshed in six hours due to a failing Snowflake Task — a ConnectionTimeout on the airline scheduling API. One pipeline failure was degrading flight recommendations across all users. I showed how, using Monte Carlo’s own intelligent autonomous observability system, we traced the issue in 12 seconds. Manual triage across dashboards, logs, and Slack threads would have taken around four hours.

That’s the difference between reactive and production-grade, and that’s what we’re helping enterprise organizations achieve.

Building agents on Snowflake Cortex is fast. Keeping them reliable at scale is a different problem — one that starts not with your prompts or your model, but with your data.

Our promise: we will show you the product.