Built for the enterprise, ready for AI: Under the hood of Monte Carlo’s platform updates

The enterprise data and AI stack is heterogeneous by design

Enterprise data stacks don’t fit into one cloud, one network, or one security review template — and they’re not going to start. Monte Carlo has 400+ enterprise customers — Nasdaq, Salesforce, and American Airlines among them — and few share the same architecture: multiple public clouds, a mix of warehouses and lakehouses, sometimes a legacy on-prem database, and an organization with strong opinions about what’s allowed to talk to what.

We’ve been building out the platform layer for years — deployment, networking, identity, governance, compliance.

Two newer pieces stand out:

• The Generic Agent, which extends Monte Carlo into on-prem and hybrid environments where our cloud-native agents can’t go.
• A composable, AI-extensible integration platform, where new connectors can be assembled from building blocks and generated through AI workflows.

Here’s a quick tour.

We named our collection agent before “AI agent” became the entire industry’s obsession. It’s the customer-side component that connects Monte Carlo to integrations — what we mean by “agent” throughout this post. The AI agents in the title? Those are what our Agent Observability product monitors. Yes, the word is overloaded. No, we’re not changing it… yet.

How Monte Carlo deploys and connects

Monte Carlo is made up of three components: the platform (the cloud service powering the UI, monitoring, lineage, alerting, AI features, and AI capabilities), a collection agent (a bridge between the platform and the customer’s integrations), and a data store (object storage holding samples of records associated with alerts and incidents).

Monte Carlo always hosts the platform. The deployment model controls where the agent and data store live, and how integrations reach the platform.

• Cloud — Monte Carlo hosts everything. Integrations connect directly to the platform by IP allowlist or private link. No infrastructure required.
• Cloud with Customer-hosted Data Store — Monte Carlo hosts the platform; the data store lives in the customer’s cloud. Integrations still connect directly to the platform. Right fit for customers who don’t want record-level data persisted outside their environment but don’t want to operate an agent. Available on AWS, GCP, and Azure.
• Hybrid: Customer-hosted Agent and Data Store — Monte Carlo hosts the platform; the agent and data store both live in the customer’s environment, and integrations connect to the platform through the agent. Maximum flexibility and control over compute, networking, and credentials. Available on AWS, GCP, Azure, and more.

Across deployment models, the platform itself runs in two regional environments — US and EU. Customers needing stronger isolation at the platform layer can opt for dedicated instances (see below).

When running hybrid, the agent itself comes in several forms. Native agents on AWS, Azure, and GCP — deployed on each cloud’s native compute and using cloud-native identity primitives. A Snowflake Native App that runs inside the customer’s Snowflake account. And our newest, the Generic Agent, for everything else.

The Generic Agent

The Generic Agent extends the hybrid model into environments our cloud-native agents can’t directly reach. It’s a containerized agent that runs on-prem, in multi-cloud setups without a clean primary cloud, and in networks that restrict inbound access from outside. Anywhere you can run Docker.

The architecture is egress-only — no inbound ports, no exposed endpoints, no firewall exceptions. The agent reaches out to Monte Carlo; the platform never reaches in.

Source credentials stay in the customer’s environment. The Generic Agent reads them at runtime from AWS Secrets Manager, GCP Secret Manager, Azure Key Vault, or environment variables. For Kubernetes deployments, secrets can additionally be sourced from Kubernetes Secrets or external stores synced via the External Secrets Operator.

For organizations with stricter networking requirements, the Generic Agent supports forward proxies, custom CA certificates for corporate TLS inspection, intermediate API gateways, and private connectivity over PrivateLink (AWS) or Private Link (Azure).

A composable, AI-extensible integration platform

Monte Carlo continues to ship native integrations for mainstream sources — Snowflake, Databricks, BigQuery, Redshift, Postgres, Oracle, SQL Server, plus other databases and warehouses, the major BI tools, the major orchestrators, and so on.

What’s new is the framework underneath. The integration layer is now a set of composable building blocks. Native connectors are one of them; Push and customer-built SQL connectors are others. A single integration can mix and match them, configured per capability.

The building blocks:

• Native connectors — Monte Carlo’s catalog, for everything we support out of the box.
• Push Ingest API — REST endpoints for pushing metadata, lineage, and query logs into Monte Carlo. Used to fill coverage gaps in an existing native integration, or to land a new source’s metadata-and-lineage layer without a connector at all.
• Custom SQL Connectors — for SQL sources whose dialect isn’t in the native catalog. A guided Claude conversation produces scaffolding, SQL templates, a test harness, and a deployable image that runs on the Generic Agent. Once deployed, the source behaves like any other Monte Carlo integration: same UI, same monitor types, same alerting, same lineage rendering.

Per-capability configuration. Each capability on a source — metadata, query logs, monitors — can be wired independently. A source might use Push for metadata and a Custom SQL Connector for monitor execution, or a native connector for some capabilities and Push for the rest. Same first-class integration surface either way.

AI-extensible. Custom SQL Connectors and Push ship with Claude skills that FDEs, partners, and customers use to deliver bespoke integrations against the same product surface. Several reference Custom SQL Connectors are public and reproducible, available to fork or to hand to Claude as worked examples.

The artifact at the end is a real Monte Carlo integration. Not a wrapper, not a sidecar.

Networking and connection security

Whatever the integration — native, custom, or Push — and whatever the deployment model, Monte Carlo supports the network and connection controls enterprise security teams expect:

• Private connectivity on AWS via PrivateLink and on Azure via Private Link, so customer integrations or agents reach Monte Carlo without exposing endpoints to the public internet.
• Network Access Control — IP-based allowlists that restrict who can reach Monte Carlo’s API, web application, and other access endpoints.
• OAuth-based authentication for sources that support it.
• mTLS and OAuth for remote agent registration.
• Self-hosted credentials for customers who prefer to keep source credentials in their own environment rather than in Monte Carlo’s secret store.
• Connection Auth Rules — a declarative framework for configuring authentication across self-hosted credential integrations. Adding a new auth type becomes a configuration change, and runs entirely inside the customer’s environment.

Dedicated instances for the most demanding environments

Beyond connection-level controls, some customers need isolation at the platform layer. Monte Carlo offers dedicated instances: each customer gets their own subdomain, isolated AWS account, and core platform infrastructure dedicated to them, with optional Azure or AWS PrivateLink connectivity for traffic.

Dedicated instances are independent of the deployment model — any of the three deployment options can run on top.

All Monte Carlo deployments include disaster recovery and high availability. Dedicated instances can also be configured for multi-region disaster recovery at the time of provisioning.

Identity, governance, and what we ingest

Beyond infrastructure controls, identity governs who can do what. Below that, ingestion controls determine what the platform sees in the first place.

On the identity side:

• SSO via SAML 2.0, integrating with any enterprise identity provider.
• SCIM provisioning for automated user lifecycle from the customer’s IdP.
• Identity-domain restriction to scope a Monte Carlo account to verified corporate domains.
• Built-in and custom roles — built-in roles cover common access patterns; an interactive policy builder supports custom roles for granular RBAC. Roles can be scoped to specific domains and assets, including monitor-type-specific permissions (table, metric, validation, custom SQL, agent monitors).
• Audit logging of user and platform actions, retrievable via API for SIEM ingestion and compliance workflows.
• Configurable session timeouts — absolute session lifetime and inactivity timeout, set per account.

On the ingestion side:

• Ingestion controls — schema- and table-level rules that determine what Monte Carlo ingests from each integration. Exclusion or inclusion rules let customers be precise about which assets are visible to the platform.
• Data sampling controls — sampling can be enabled or disabled at the integration level, or restricted granularly per database, schema, table, or tag. Customers who don’t want record-level data persisted at all can configure that explicitly.

Compliance and security posture

For the security and compliance teams reviewing Monte Carlo, the baseline is what you’d expect from a platform built for regulated industries.

Certifications and attestations: SOC 2 Type 2 (audited annually), ISO 27001, ISO 27017, and ISO 27018, plus annual third-party penetration testing. Monte Carlo’s infrastructure controls also adhere to GDPR and CCPA.

Architecture: the platform follows AWS Well-Architected and CIS Benchmark guidelines, with AWS WAF in front of platform endpoints, network segmentation across production, staging, and development, and immutable logs supporting audit and forensic analysis.

Encryption: TLS 1.2 or higher in transit, AES-256 at rest across all data stores and backups. For AWS customer-hosted agents and data stores, customer-managed encryption keys (KMS) are supported for those components.

Operations: a 24/7 on-call security and engineering team responds to alerts and incidents under established containment and recovery procedures.

Attestations, reports, and the Shared Responsibility Matrix are available via Monte Carlo’s Trust Center.

What this adds up to

The pieces above are aimed at one thing: making it possible for a large, regulated, multi-cloud, partly-on-prem enterprise to adopt Monte Carlo without compromising on where data lives, how it’s accessed, or who controls it.

If you’re evaluating data and AI observability in a regulated industry, a multi-cloud environment, or an on-prem deployment, visit our Trust Center for the security and compliance details, or talk to us about your specific architecture.

Our promise: we will show you the product.