The Open vs. Closed AI Debate Misses the Point: Most Orgs Cannot Measure Either

The discussion around enterprise AI often focuses on the choice between open-weight models, like Meta’s Llama and models from Mistral, vs. closed frontier models, such as Anthropic’s Claude and OpenAI’s GPT models. 

While closed frontier models promise state-of-the-art reasoning without managing infrastructure,  open-weight models promise sovereignty, portability, and freedom from vendor lock-in. Public commentary has typically framed the debate as an ideological decision—democratized AI versus corporate gatekeeping. The reality, however, is much more nuanced, and in June 2026, G7 Digital and Technology Ministers formally rejected the binary framing in favor of a four-tier spectrum.

Production data published from different operators in the space actually reveals that companies are not picking a single side. In July 2026, for example, open-weight models accounted for 36% of tokens routed through Vercel’s AI Gateway on 8.6% of spend— up from 29% in June. In that same month, Anthropic alone represented roughly 65% of gateway spend on 30% of volume. 

The truth is that enterprises are running both open and closed models and routing tasks by workload type. This is typically done by directing high-volume, routine operations to open weights and high-stakes tasks to frontier models.

Looking at the rate of change also highlights another movement, one where open weight models are also being increasingly trusted to do more complex work. 

Open-weight token volume more than tripled over four months, while spend share actually grew faster: in one month it rose from under 4% to 8.6%. If open weights were only being used to shave costs on throwaway work — bulk classification, summarization, cheap batch jobs — volume share would climb while spend share stayed flat. The fact that we are observing the opposite suggests that companies are increasingly trusting open weights with work that matters, not just cheap filler. 

Volume Share and Spend Share of Open-Weight Models:

The operational split of using open weights for routine tasks and frontier models for high-stakes work makes practical sense, but it introduces other engineering requirements. When a single application relies on multiple models from different vendors, each with distinct licenses and release cadences, the challenge shifts from selecting the best model to accurately monitoring its performance in production. An unmonitored routing setup introduces a lot of risk to the system. 

Open versus closed represents a real decision with economic, legal, and operational tradeoffs. However, each of these factors leads back to the same fundamental requirement: system visibility.

You don’t have to be married to your model

Model selection isn’t a permanent commitment, but rather a per-workload routing decision that should be re-evaluated regularly.

When we look at production data of AI systems, it reveals a consistent pattern about this decision. 

During initial exploration—such as prototyping an agentic workflow, a document synthesis pipeline, or a complex extraction task—speed and raw capability take priority. Paying frontier API rates to validate ROI in three weeks rather than three months is standard practice, as infrastructure overhead at this stage only creates friction.

Once a use case reaches a steady state—such as high-volume classification, extraction, routing, or internal Q&A—the priorities flip. The task becomes well-defined, the quality baseline is established, and paying a premium for frontier reasoning is no longer necessary. This is where token volume shifts toward open weights for many organizations.

However, the distribution across companies tells a more specific story than overall volume averages:

Comparing these data points brings the reality into focus. Roughly two-thirds of organizations are using open source somewhere in their AI stack, but only about one in eight run open-weight models in production. Notably, Menlo’s 11% figure and Vercel’s 12.5% come from entirely different methodologies—one a survey of enterprise spend, the other production routing telemetry—yet land within 1.5 percentage points of each other.

Another factor is this: the volume of open-weight model usage actually sits within a small subset of companies. Strategies assuming that model distillation and custom hosting are widespread, therefore, should account for how few organizations have actually implemented them, especially given that Menlo measured open-source enterprise share dropping from 19% to 11% even as open models improved dramatically. Model capability does not seem to be the limiting factor.

Rather, the big risk in shifting from a frontier model to an open-weight alternative is undetected quality loss. This migration is only safe if you can verify that the cheaper model performs equivalently on your specific data—not just on standard benchmarks like MMLU. That requires a held-out evaluation set, workload-level quality telemetry, and side-by-side traffic testing. Routing without these checks risks degrading system outputs under the guise of cost savings.

The Economics: Three Options, Not Two

Arguments for open weights often start with a simple premise: why pay per token when the model weights are free? The standard counter-argument—that hosting GPU clusters is expensive—is equally simplified, as it compares the two most extreme setups while ignoring how most teams actually deploy.

There are three primary deployment models (frontier model examples use standard synchronous pricing):

Frontier pricing sources: Anthropic, OpenAI, and Google August 2026

Hosted open weights (the middle option) are frequently overlooked in cost comparisons. Services like Together AI host Llama 3.3 70B at $1.04 per million tokens, Fireworks at $0.90, and Amazon Bedrock offers comparable pricing. This makes open-weight models an order of magnitude cheaper than frontier APIs, without managing infrastructure, procuring GPUs, or absorbing utilization risk. For many enterprise workloads, this represents the most practical setup.

True self-hosting has a very high burden and should be evaluated as a distinct operational decision based on specific workload needs. Self-hosting swaps a variable cost for a fixed one, but fixed costs only win at high sustained volume, and only if you keep the asset busy. 

An OECD study from May 2026 mapped payback periods across volume tiers, comparing self-hosting with your own hardware against pay-as-you-go pricing on a low-cost closed-weight API:

Consider the practical scale of these somewhat abstract numbers: a customer support workflow processes 300 million tokens monthly, which amounts to roughly 5,000–20,000 support conversations per month, or a few hundred a day. Even at that volume, it remains far below the self-hosting break-even threshold, making hosted APIs the economical choice even if the weights are free. You’d need to be somewhere north of a few billion tokens a month before the math turns in favor of self-hosting.

Below the volume threshold, therefore, self-hosting is primarily a data sovereignty decision rather than a pure cost optimization.

Licensing considerations of open weight models

Non-technical decision-makers building AI programs often conflate open-weight models with true open-source software. These are not the same, and downloading model parameters does not automatically grant you unrestricted commercial rights to the model.

Licensing terms vary significantly. For example, Z.ai’s GLM 5.2 uses an MIT license with no usage caps or domain restrictions. Conversely, Meta’s Llama Community License allows commercial use but requires a separate agreement past 700 million monthly active users, prohibits using outputs to train competing models, and incorporates Meta’s acceptable-use policies by reference—allowing terms to update over time.

Model licensing, therefore, has to be treated as a standard legal and governance check for every model and version deployed.

The importance of visibility into the system

There is an underlying issue here and it is squarely related to AI visibility: unit cost metrics are only helpful when properly attributed. Many organizations struggle to trace which specific workloads consume which models, at what cost, and for what quality output. As a result, cost reduction often focuses on negotiating API rates rather than eliminating inefficient usage—such as excessive context length, unnecessary retries, or routing simple tasks to expensive models. Identifying these inefficiencies requires granular, workload-level telemetry.

Your Data Is Both the Asset and the Liability

The big advantage of open weights is the fact that organizations can maintain long-term control over proprietary data and operational traceability. For enterprises in regulated industries where compliance is paramount and customer data is highly sensitive (e.g. financial services, healthcare) this is a considerable benefit. 

What open weights provide

Verifiable data boundaries

For financial, healthcare, and defense applications, routing sensitive data through external endpoints creates compliance challenges that contracts alone may not fully address. The sovereign cloud market reached roughly $80 billion in 2026, growing 35.6% year-over-year, driven largely by regulated AI workloads moving to localized infrastructure. Similarly, 40% of enterprise leaders surveyed by McKinsey expressed a preference for self-hostable models due to privacy concerns.

Permanent model assets via fine-tuning

If you fine-tune a closed frontier model, the result sits on the vendor’s servers, governed by their terms. But fine-tune an open-weight model and you get a permanent file — your domain knowledge baked into weights you keep.

Portability and vendor leverage

Owning model weights enables migration across hosting environments (such as Bedrock, Together, Fireworks, or self-hosted hardware) to manage costs or mitigate outages. This operational flexibility provides meaningful leverage during vendor negotiations.

This operational risk actually reared its head in real life in June 2026. Following the release of Anthropic’s Claude Fable 5 on June 9, adoption scaled quickly. But on June 12, new US export control directives affected the model, leading to a temporary suspension of service through the end of the month. 

As you can see, model availability can be influenced by regulatory shifts as well as infrastructure uptime, meaning single-endpoint architectures carry inherent operational risk.

What open weights don’t provide

Data isolation is not system visibility

Hosting a model within a private cloud ensures your data doesn’t leave your perimeter, but it does not track what data enters the model, its source, its freshness, or whether it should have been included in the prompt. 

An isolated model connected to an unmonitored retrieval pipeline still lacks governance, and regulatory reviews are increasingly focused on tracing input data and verifying model behavior, which infrastructure boundaries alone do not address. 

Fine-tuning permanently embeds data quality issues

If a retrieval pipeline supplies incorrect data to an API, the system produces a single bad response, which is then resolved by fixing the pipeline. However, if bad data feeds a fine-tuning run, errors—such as outdated records, duplicate entries, or unverified labels—become baked into the weights themselves, requiring retraining to fix. Fine-tuning only creates a valuable asset if the underlying data is clean and well-governed.

Supply chain considerations

Using open-weight models introduces open-source software dependencies. For example, Python’s pickle serialization format, historically common for model distribution, can execute arbitrary code during loading. Additionally, namespace hijacking allows malicious actors to claim inactive maintainer names to publish altered model files. In February 2026, security researchers identified 341 malicious entries in an agent-skill registry distributing malware. Managing open-weight models requires checking artifact hashes, tracking provenance, and maintaining an updated bill of materials for models, just as with traditional software dependencies.

Data and AI leaders are, rightfully, very worried about this, as is evident in a recent McKinsey survey. According to the study, enterprise respondents flagged cybersecurity (62%), regulatory compliance (54%), and IP protection (50%) as major concerns when deploying open models. These are all major operational responsibilities that shift to internal engineering and security teams when organizations decide to adopt open weight models. 

The core challenge is this: open-weight models do have a number of very clear benefits, but these depend on rigorous operational verification. 

Model portability requires proof that a replacement model matches the performance of the original across target tasks, while data sovereignty requires clear audit trails of inputs and outputs. Without established evaluation frameworks, tracing pipelines, and production testing, switching models or environments can take months rather than days, introducing a level of risk that enterprises need to carefully manage. 

This is where system-level observability comes into play.  

Operational Requirements for Production AI

Routing across models, managing costs, and maintaining deployment flexibility all depend on the same baseline capability: system observability.

This becomes increasingly important as the role of AI shifts from generating text to actually  executing automated actions. As AI agents handle production workflows, you need to have visibility across four key areas to ensure true system-wide trust:

Context — garbage in, autonomous garbage out.

High-performing models will execute incorrect actions if supplied with stale, duplicated, or inaccurate context. System reliability depends heavily on upstream data quality, vector database maintenance, and context freshness before a prompt ever reaches the model.

Performance — the routing decision, measured.

Multi-model architectures turn cost and latency into per-workload variables rather than fixed properties of the system. Knowing which model served which request, at what latency and what token cost, is what makes routing a decision you can defend instead of a guess.

Behavior — what the agent actually did.

Managing autonomous systems requires visibility into every step of an agent’s execution path: reasoning chains, tool calls, retrieval results, and source data lineage. Behavior also shifts underneath you from both directions: closed APIs get provider updates that change how a system responds, while a pinned open-weight version prevents surprise updates but drifts out of step as the data and requirements around it move.

Outputs — was it right?

Clean context, healthy performance, and a complete trace still don’t tell you the answer was correct. Without evaluation against ground truth, quality regressions arrive as customer complaints rather than alerts.

Strategic Framework

The question for enterprises as they build out their AI architecture is not a binary of open vs. closed, but rather how you can build a flexible, sustainable system that you can observe and adjust as needed. 

Model selection is ultimately a recurring decision, not a static one, and it changes based on what you need your agents to do, the scale at which they will act, and the operating burden your team is able to manage. Here, I’ve laid out a strategic framework to guide this process: 

• Evaluate models on proprietary data. Benchmark leaderboards do not reflect operational performance within specific enterprise systems. You should be testing models on internal datasets and real-world edge cases.
• Match models to workload requirements. Use frontier models where complex reasoning and fast iteration are required, but consider shifting to hosted open-weight models for high-volume, well-defined tasks. Self-hosting is a significant endeavor with huge operational overhead that should be reserved for strict sovereignty mandates or volumes that justify dedicated infrastructure.
• Establish legal and security checks early. Review licensing terms for every model and version prior to deployment. Verify checksums, document artifact provenance, and track model dependencies across internal registries. 
• Measure operational switching costs. Assess how quickly your team can validate and ship an alternative model. True vendor independence depends on how efficiently alternative models can be evaluated and deployed.
• Build observability into the architecture. This is critical. You should implement telemetry across data pipelines, model inputs, tool interactions, and system outputs as early as possible. Verifying data quality and lineage upstream prevents propagating errors into fine-tuned models and downstream agent actions.

Sustainable advantages come from robust data pipelines, governance frameworks, and observability systems built around model infrastructure. Models will continue to evolve and pricing structures will shift, but maintaining a flexible system architecture allows teams to adapt efficiently alongside the advancing tech.