Investment Thesis for AI: Valuing Intelligence in the Age of Inference Arbitrage

Let me tell you something nobody in Silicon Valley wants to say out loud at a dinner party.

The valuation frameworks that made people billionaires between 2020 and 2024 are fundamentally broken.

The SaaS playbook was elegant. Build a product. Charge a subscription. Your marginal cost of serving another customer was basically zero. Gross margins hit 80 to 90 percent and that pattern justified 10x, 20x revenue multiples. Everyone understood the math.

Then AI arrived and turned that math inside out.

When your product core is a model API call, every extra customer costs you something real. Electricity. GPU time. Memory bandwidth. You cannot subscribe your way to SaaS margins when your COGS is measured in tokens per second.

This is not a temporary disruption. It is a structural shift in how intelligence is produced, distributed, and valued. And if you are building or investing in AI companies in 2026, you need a completely new mental model.

The End of SaaS Multiples for AI-Native Products

Here is the uncomfortable truth. Companies that built their entire business on top of a single frontier model API are walking into a margin crush.

In 2023, an AI startup could build a customer service chatbot, charge $200 per month per enterprise client, and pay the model provider$30 per month in API costs. That was a sustainable business. The gross margins were good enough. The valuation multiples justified the spend.

But the model providers have been in a race to the bottom on inference pricing for two years. Every new model generation cuts per-token costs in half. A distilled 7B model now matches the quality of what was considered “frontier” 18 months ago, at a fraction of the cost.

The result? Your API cost that was $30 last year is now$3. And the customer who was paying $200 per month? They are seeing competitive offerings at$80 per month. The margin compression is unavoidable.

This is not unique to customer service. Every category of AI-native product built on top of open model APIs faces the same structural pressure. Innovation cycles are collapsing because anyone can spin up a competing product using the same underlying models.

Inference Arbitrage: The New Profit Model

So if you cannot charge subscriptions with thin margins and you cannot build a moat around model quality, where does the profit come from?

The answer is inference arbitrage. And I use that term deliberately because it captures the mechanics more accurately than buzzwords like “model routing” or “intelligent orchestration.”

Inference arbitrage means building systems that dynamically route requests between different models based on what each task actually needs. The simplest case is trivially profitable. A customer support ticket about a billing question does not need your $0.50-per-call GPT-5. It needs a$0.002-per-call distilled model that handles those queries with 94 percent accuracy.

The difference between routing that billing question through a frontier model versus a distilled model is not incremental savings on your operating statement. It is the difference between a profitable business and an unprofitable one.

But the real opportunity is more sophisticated than a simple rule-based router.

The companies that will win at inference arbitrage are building systems that can evaluate, in real time, the quality-cost tradeoff for every single request. They build confidence estimators that know when a distilled model’s answer is close enough and when the risk of error demands escalation to a more expensive model.

Think about what this means for valuation. A company that routes 80 percent of its requests through discounted models and only uses frontier models for the 20 percent that genuinely need them has fundamentally different unit economics than a competitor that sends everything to the most expensive model available.

The routing company might even charge less than the competitor and still maintain higher margins. That is a structural competitive advantage that no marketing budget can overcome.

The Open Weights Shock to Model API Valuations

This brings me to the most disruptive force in the AI market right now.

Open weights models have destroyed the moat that model API providers built over the past three years.

For years, the narrative was clear. The companies building the biggest models had a defensible advantage. You could not replicate their results. You had to pay them. That created a pricing power that justified enormous valuations for organizations like Anthropic, Cohere, and various Chinese model providers.

Open weights changed that equation entirely.

When Llama, Mistral, and now open sources of comparable quality become available for download, the model API provider’s advantage shrinks to two things: convenience and scale. Convenience for teams that do not want to run their own inference infrastructure. Scale for organizations that serve millions of concurrent requests and need the kind of optimization that only comes from operating at that volume.

Convenience is real but it is shrinking. The inference infrastructure stack has matured dramatically. Serving a distilled model on your own GPU cluster is easier today than building a Slack bot was in 2015.

Scale advantages are more durable but their economic value is smaller than most people think. The inference optimizations that a company like OpenAI has baked into their serving infrastructure at scale probably save them 20 to 30 percent on serving costs compared to a well-engineered competitor. That is meaningful but it does not justify monopoly margins in a market where the foundational model weights are freely available to anyone.

The valuation implication is stark. Model API providers that lack a distribution moat or proprietary data advantage are worth significantly less than the market priced them at in 2023 and 2024. The arbitrage window between their pricing power and their actual cost is collapsing.

This does not mean model building is worthless. It means the economic rents are shifting away from the model layer itself and toward the layers above and below it.

Where the Moats Actually Are in then?

If you strip away the hype and look at where real economic value is being captured in the AI stack, three categories emerge clearly.

First is data distribution. The companies that have direct access to real users, real workflows, and real feedback loops on their product. OpenAI has ChatGPT. Google has Search. These are distribution moats that no open weights model can replicate. The model quality catches up eventually but nothing replaces the feedback loop of millions of daily users stress-testing your product in the wild.

This is why organizations that combine a proprietary user-facing product with AI capabilities tend to have stronger long-term positions than pure model API playes without distribution. They own the relationship with the customer. The model becomes a cost center to optimize rather than a revenue center to defend.

Second is specialized infrastructure. The companies building the tools that make inference cheaper and faster. These are organizations working at the intersection of hardware and software. The companies that can serve a model for $0.001 per token that would otherwise cost$0.01 are building genuinely valuable businesses.

The infrastructure layer includes everything from specialized inference servers to compilation toolchains to network architecture optimizations. Each of these creates real economic value that flows downstream to every company built on top of them.

Third is vertical integration in narrow domains. The companies that understand a specific industry deeply enough to build models and workflows that generic AI cannot replicate without massive customization. Healthcare diagnostics. Legal contract review. Medical imaging analysis. These domains combine specialized data, regulatory compliance requirements, and workflow integration that creates genuine switching costs even when the underlying model technology is commoditized.

The Valuation Framework for AI Companies in 2026

So how do you actually value an AI company now?

The metrics that mattered in 2023 need to change. Monthly active users still move the needle but they are less meaningful when your user base is interacting with a product whose cost structure is fundamentally different from traditional software.

What matters more is inference efficiency per dollar of revenue. How much compute does it cost you to serve each customer? How efficiently are you routing between model tiers? What percentage of your requests need frontier-level capability versus what can be handled by distilled models?

These metrics map directly to gross margin trajectory. A company that is improving its inference efficiency by 30 percent per quarter while growing revenue at 15 percent per quarter has a fundamentally different investment profile than a company growing revenue but seeing its average inference cost per customer rise quarter over quarter.

Another critical metric is data flywheel velocity. How quickly does your product generate proprietary data that improves your competitive position? A customer service AI that logs every conversation and uses it to fine-tune a domain-specific model becomes more valuable over time in ways that a competitor building from scratch cannot match, regardless of which model they choose.

Open weights models may democratize intelligence but they do not democratize data. The organizations that combine product-market fit with efficient inference architecture and compounding data advantages are the ones that deserve premium valuations in 2026.

Everything else is a subscription service waiting for the margins to get compressed.