Building software is getting cheaper; the cost of getting it wrong is skyrocketing: a chat with Nasdaq VP

Yesterday at Databricks Data + AI Summit, I had the chance to sit down with Lenny Rosenfeld, VP of Data Science and Software Engineering at NASDAQ, for a fireside chat about what it actually takes to deploy AI agents in one of the world’s most regulated, most consequential data environments.

NASDAQ isn’t experimenting with AI – far from it. They’re running it. And the lessons Lenny shared — about trust, governance, the organizational reality of change, and what it means when AI fails silently — are ones every data and engineering leader building with agents right now needs to hear.

AI fails as a whisper

Lenny expressed something that most data and AI practitioners experience constantly: that AI failures are silent, insidious, and dangerous: 

“AI doesn’t fail spectacularly — a server blowing up or code that just stops working. It fails silently, which is really dangerous. You might not even know it failed. Or it’s just lying to you, and it looks really right.”

This is the fundamental observability challenge of the agentic era. Traditional software breaks loudly. You’ll notice when pipelines go down or errors surface, oftentimes immediately because someone on the team gets paged. But an AI agent that hallucinates a confidence score, mismatches an entity, or quietly returns a plausible-but-wrong answer? That can travel deep into downstream systems before anyone notices.

This is why NASDAQ built their entity resolution agentic system — which maps and matches thousands to hundreds of thousands of data entities across sources — with humans in the loop at every verification step, at least for now. The goal is to reach a confidence threshold where the agent’s reasoning is sound enough to pass through without human review. But they’re not there yet, and Lenny is clear-eyed about why that matters: at NASDAQ, the data product ‘is’ the reputation.

Governance embeds within the product, it doesn’t block it

About 18 months ago, NASDAQ established an AI governance committee. Every use case — internal or external, product or productivity — goes through it. Vendors, models, data classification, and validation methodology are all evaluated before a single line of agentic code ships.

That might sound slow, but it isn’t really – or at least it doesn’t have to be. Lenny described how the process has sped up significantly as patterns have emerged. More importantly, it’s forced the right questions:

• Does this use case make sense for the business owner, regardless of the fact that we’re using AI?
• How do we validate outcomes? How do we prove it’s doing what we said it would do?
• Can we explain what the agent did to regulators? To compliance? To our own staff?

This last question, explainability, came up repeatedly. In a regulated environment, “it worked” isn’t enough. You need to be able to trace the reasoning, document the outcome, and defend the decision. That’s not just a NASDAQ requirement; as agents proliferate in financial services, healthcare, and enterprise software broadly, explainability will become table stakes.

The decision framework: where do humans step back?

One of the most useful frameworks Lenny shared is deceptively simple: “Is the workflow deterministic?”

If the process is well-defined, well-understood, and executes the same way every time — that’s a strong candidate for automation. The goal isn’t to replace human judgment; it’s to remove humans from tasks that don’t require it, so they can be deployed on problems that do.

At NASDAQ, AI agents can build software. They can write documentation and tests, as well as ingest, normalize, and validate incoming data products. But they do not deploy to production; that gate stays with humans.

For higher-stakes decisions — like the alternative data products that generate alpha signals for investment strategies — every output is validated by data scientists and finance professionals because those outputs carry reputational risk. 

As Lenny said, “At the end of the day, that is reputation. That is what we’re selling.”

Orchestration and context are differentiators

NASDAQ is already running fleets of specialized agents — one to check infrastructure (is this an AWS issue?), one to check data quality (is this a pipeline issue?), one to interface with Monte Carlo for data observability, and then others to diagnose and remediate. The orchestration layer routes context to the right agent at the right time.

What makes this work isn’t the models, since models will keep improving and swapping out. The context layer is what’s most critical: your company’s proprietary logic, the instructions that enforce your standards, and the institutional knowledge that tells an agent what “correct” means in your environment.

Lenny put it plainly: “The promises, the context, the tooling, the skills, the instructions — that’s the glue that allows you to chain frontier models with your proprietary logic. That’s the core of it.”

For practitioners thinking about how to future-proof their agentic stacks: the models are commoditizing, but the context isn’t.

Validation needs to scale faster than built times

One of the most important observations from our conversation is this inversion: the cost of building software is dropping, but the cost of getting it wrong is rising.

When agents can write code, the bottleneck is no longer generation, but verification. For every prompt that produces a piece of software, Lenny suggested it should take five prompts to validate it actually meets the metrics and SLAs you care about.

This means testing, observability, and quality assurance become even more important in an agentic world. 

Culture is a slow mover

NASDAQ’s goal this year is for everyone in the company — not just engineers or data teams, but product, executives, and compliance — to be educated on AI. Everyone needs to understand it well enough to ask good questions and make good decisions, even if they are not actively building and deploying agentic solutions themselves. 

Lenny was honest about the organizational reality; there are enthusiasts and there are skeptics. His advice for the skeptics isn’t to push harder, but to listen harder. One of his engineers said he’d stayed for 20 years because he loved writing code. The response wasn’t “agents will write your code now.” It was: “Are there other parts of your job you hate? Let’s start there.”

That’s the change management insight that will determine which organizations actually get value from AI and which ones spin in place. 

The new contract between humans and agents

Lenny clearly articulated the new contract between humans and AI systems. It’s not about replacement, but rather redeployment.

Agents handle the deterministic, the repetitive, the high-volume, the cognitively expensive-but-low-stakes. Humans focus on the novel, the nuanced, the high-accountability. The goal isn’t fewer humans — it’s more complex, creative, and meaningful work for humans, with systems underneath them that they can actually trust.

But trust doesn’t emerge automatically from capability. It has to be built through governance, observability, explainability, and validation. In a world where AI fails silently, the infrastructure of trust is foundational to everything else.

Our promise: we will show you the product.