GAVIN BAKER: Lower model-layer margins raise demand for compute because similar tokens use similar compute while cheaper tokens are marked up less

The Gist

Gavin Baker says a token from a cheap open model takes about as much computing as one from an expensive closed model. Cheap tokens just leave less markup for the model company, so more of the money goes to chips and clouds. Rising compute prices fit that story. This is a steelman: Gavin Baker's thread is reconstructed in its strongest fair form, not endorsed.

Conclusion

As long as someone funds training, lower margins at the model layer mechanically shift more of each end-user dollar to compute and raise compute demand, so broad open-weight share is the best case for AI infrastructure demand and a competitive field of several frontier labs is the next best.

Premises

1. Tokens from leading closed models and leading open-weight models of similar size use roughly the same compute and energy to produce.
2. Closed frontier tokens carry an estimated 75 to 80 percent inference gross margin, while open-weight tokens carry roughly 30 to 50 percent.
3. For the same underlying compute cost, a closed frontier token produces about four to five times its cost in model-layer revenue, while an open-weight token produces about two times.
4. Open-weight providers have less buying power than the leading labs, so clouds and neoclouds should earn more per token serving them.
5. Lower prices to end users raise token volume, so a larger share of each end-user dollar reaches compute.
6. Compute prices have risen as the model layer became more competitive, and inference clouds focused on open weights are reported to bid well above the roughly $50 per watt implied by the Google and SpaceX deal.

Assumptions

• The margin figures are Baker's estimates, not disclosed numbers.
• The roughly $100 per watt bids are described as alleged and could not be confirmed; the roughly $50 per watt reading of the Google and SpaceX deal comes from analyst back-calculations.
• GPU rental data were mixed in 2026: Silicon Data indices rose into October, while another tracker recorded a sharp B200 decline after a late-May peak.
• Rising rental prices partly reflect scarcity of power and buildings; one analysis notes hyperscalers pay neoclouds a premium only while their own capacity lags.

Analysis

Argument type: Inductive.

Potential Fallacies

• Conflating share of wallet with absolute demand (P5 to Conclusion) — A larger share of each end-user dollar going to compute raises compute demand only if the total dollar pool holds or grows, and if compute per token does not fall. If open weights simply displace closed tokens one-for-one at equal compute, demand is unchanged. If demand is inelastic, total spend shrinks. The identity supports the share claim, not the demand claim.
• Overstated certainty ('mechanically') (Conclusion) — The language of mechanism attaches to a conclusion that depends on empirical elasticity, efficiency, workload mix, and training funding. Only the share-of-dollar effect is truly mechanical. The 'best case' and 'next best' ranking is a forecast presented as a settled hierarchy.
• Correlation read as causation / price signal read as demand signal (P6) — Compute prices rising while the model layer became more competitive is offered as support for the mechanism. The stated assumptions point to rival explanations: scarcity of power and buildings, hyperscaler capacity lag, and a general AI demand surge. A conflicting tracker also shows a B200 decline. Price rises from supply limits do not show that lower model margins caused higher demand.
• Conditional that carries the weight (treating an endogenous variable as exogenous) (Conclusion) — 'As long as someone funds training' sets aside the strongest counter-consideration. Funding for training is partly paid by the very margins that the mechanism compresses. The likelihood that funding persists is never assessed, so the claim is conditionally true but practically uncertain.
• Non-independent premises (double counting) (P2 and P3) — P3 is an arithmetic restatement of P2, so it adds no independent evidential weight. The figures are also slightly loose: a 30% margin implies about 1.4x, not 2x, so 'about two times' fits only the upper end of the open-weight range.
• False precision and reliance on a single authority (P2, P3) — Point multiples such as 4-5x versus 2x are derived from wide, unverifiable estimates by one commentator with a plausible interest in a compute-bullish thesis. The authors flag this as an assumption, but the conclusion's firmness does not reflect it.
• Unsupported leap from comparative to superlative (Conclusion) — The premises, if granted, indicate a direction (lower margins mean more compute share). They do not rank scenarios against each other or against other drivers such as agent adoption, enterprise uptake, or training scale-up.

Counterarguments

These challenges are not ranked.

• Conclusion (High impact) — Margin is a transfer, not a source of demand. Compute demand equals tokens times compute per token, plus training. If token demand is inelastic or saturates in major segments (seat-priced enterprise tools, fixed workflows), lower prices shrink total spend and compute demand could stay flat or fall even as compute's share of each dollar rises.
• Conclusion (High impact) — The funding condition is endogenous. If open-weight dominance squeezes closed-lab margins, labs lose the cash flow for frontier training and the large compute commitments that are among the biggest marginal buyers of compute. Open-weight releasers are themselves subsidized by corporate or state strategy that may not persist. Training demand and capability-driven demand growth could fall by more than the inference share gains.
• Conclusion (Medium impact) — The ranking could reverse for training demand. A concentrated market with 75-80% margins could fund far more training compute than a commoditized field. A concentrated-lab bull case, with long-term capacity commitments from well-capitalized buyers, is a serious alternative that the argument never engages.
• Premise 5 (High impact) — Cheaper open-weight tokens drive distillation, quantization, smaller models, and on-device or on-prem inference, which could cut compute per unit of value faster than volume grows and move workloads away from hyperscale GPU clouds.
• Premise 6 (Medium impact) — Compute price rises are explained by power and building scarcity, hyperscaler capacity lag, and a general surge in agentic and reasoning demand. The sharp B200 decline after the May peak shows prices follow supply cycles. High prices may later attract overbuild, as with the fiber buildout of 2000-2002, and a price collapse.
• Premise 1 (Medium impact) — Closed labs with custom silicon (TPU, Trainium), tuned kernels, and high utilization may have materially lower compute cost per token than open-weight providers on rented GPUs, so the cost basis is not identical and the 4-5x versus 2x comparison is not like for like. Closed tokens may also deliver more value per token, which would justify part of the margin gap.
• Premise 4 (Medium impact) — Open-weight serving is a commoditized, competitive market with low entry barriers, so cloud margins should compress toward cost rather than rise. Margin shifted to compute may also accrue to chip vendors, power, and real estate rather than raising physical compute deployed.
• Conclusion (Low impact) — Closed labs may cut prices or margins in response to open-weight competition. That produces the same share shift without broad open-weight share, which weakens the distinction between the 'best' and 'next best' scenarios.

Suggested Improvements

• Separate share from level — State the conclusion in two parts: (a) lower model-layer margins raise compute's share of each end-user dollar, and (b) total compute demand rises only if price elasticity exceeds a stated threshold. Give the break-even elasticity for the author's margin figures. This keeps the strong accounting result while making the empirical dependency explicit, and it replaces 'mechanically' with a testable condition.
• Endogenize training funding — Model how training budgets respond to margin compression, and say who funds open-weight releases (Meta, Chinese labs, sovereigns) and how durable that is. Include training compute in the comparison of scenarios. The funding clause is the argument's keystone. Treating it as endogenous addresses the strongest objection and lets the scenario ranking be tested on total compute, not inference alone.
• Quantify and source the margin figures — Define 'inference gross margin' (amortization, utilization, free-tier treatment), give ranges with a method or cross-check from public pricing and throughput, and present revenue multiples as ranges consistent with the margin bands (for example, 1.4-2x for 30-50%). This removes false precision, fixes the small inconsistency in P3, and lets readers see how sensitive the conclusion is to the estimates.
• Isolate the demand signal from the scarcity signal — Support the compute-pricing claim with measures less affected by supply constraints, such as open-weight share of tokens against compute spend and utilization, with controls for power and capacity limits. Give the contrary tracker equal weight and say what price pattern would falsify the thesis. It would reduce confirmation-bias concerns and show that the mechanism, not scarcity alone, is driving the data.
• Account for efficiency and substitution — Add compute per token and tokens per task as explicit variables, covering distillation, smaller models, on-device inference, and the countervailing rise in agentic and long-context use. Net compute demand is volume times compute per unit. Tracking only volume and share leaves out the offsetting terms.
• Define 'best case' and 'compute demand' — Specify whether demand means spend, physical capacity, or pricing power, and for whom the scenario is 'best'. Add falsification criteria and temper the language to reflect conditional probabilities. Clear terms make the ranking testable and stop every market structure from being read as bullish.

Scenario Tests

• Token demand is highly elastic (elasticity well above 1) and compute per token stays roughly constant as open-weight share grows. (Supports) — The share shift compounds with volume growth and both the compute share and absolute compute demand rise. This is the world the argument describes.
• Token demand is inelastic or saturating in major segments, such as seat-priced enterprise tools or fixed workflows. (Challenges) — Lower prices shrink total end-user spend. Compute's share rises but absolute compute dollars flat-line or fall, and the 'mechanical' conclusion fails.
• Margin compression starves frontier training funding, and open-weight releasers (Meta, Chinese labs) scale back or are restricted. (Challenges) — Training compute falls, the open-weight pipeline dries up, and capability gains stall. The funding condition fails because of the mechanism, and the long-run demand growth case weakens.
• Algorithmic and hardware efficiency (distillation, sparsity, quantization) reduces compute per token faster than token volume grows. (Challenges) — Net compute demand falls despite more tokens. Broad open-weight use could also move inference to cheaper silicon or on-premises hardware.
• Cheaper tokens unlock agentic, long-context, and reasoning-heavy workloads that consume far more compute per task. (Supports) — A channel the argument does not name could raise compute demand beyond what the share shift alone implies, strengthening the directional claim.
• Compute supply (power, buildings, chips) is the binding constraint through the period. (Neutral) — Extra demand shows up as higher prices and scarcity rents, not more physical compute. The claim is consistent with rising prices but cannot be separated from the scarcity explanation, and gains go to holders of scarce inputs.
• Capacity catches up to demand, and neocloud and hyperscaler premiums erode, as one analysis of the data anticipates. (Challenges) — Present price strength fades and the P6 evidence turns out to reflect a temporary regime. The structural claim would need support from volume and utilization data instead.
• Several frontier labs compete on price without a large open-weight tier. (Supports) — Margin compression still shifts share to compute, consistent with the 'next best' ranking. But the effect on training funding differs, so the ordering depends on how far prices fall.

Coherence & Relevance

The argument is internally coherent on its core accounting point: with equal compute cost per token, lower model-layer margins leave a larger compute share of each end-user dollar. The weaker links come where it moves from share to absolute demand and then to a ranking of market structures. That move needs elasticity, continued training funding, stable compute per token, and compute supply that can respond, none of which the premises establish. The empirical premise (P6) is confounded, and the funding clause conceals an interaction between the conclusion's mechanism and its own condition. The thesis is best read as a plausible, conditional directional claim favoring compute's share of value, not a mechanical guarantee of higher compute demand or a proven ranking of scenarios.

• Tokens from leading closed models and leading open-weight models of similar size use roughly the same compute and energy to produce. (Strong) — Necessary for treating the margin gap as a pure markup difference. 'Similar size' does not guarantee equal compute, and equal compute does not mean equal value delivered.
• Closed frontier tokens carry an estimated 75 to 80 percent inference gross margin, while open-weight tokens carry roughly 30 to 50 percent. (Strong) — Directly sizes the shift. Estimates, wide ranges, and undefined cost allocation limit how much of the shift can be quantified.
• For the same underlying compute cost, a closed frontier token produces about four to five times its cost in model-layer revenue, while an open-weight token produces about two times. (Moderate) — Redundant with P2 and slightly inconsistent at the low end of the open-weight range. It restates rather than extends the argument.
• Open-weight providers have less buying power than the leading labs, so clouds and neoclouds should earn more per token serving them. (Moderate) — Supports the claim that compute captures more value but sits uneasily with the equal-cost basis of P1. It shows a transfer to clouds, not growth in compute demand, and ignores competition among clouds.
• Lower prices to end users raise token volume, so a larger share of each end-user dollar reaches compute. (Strong) — This is the bridge from share to demand, but it supplies only the direction of the volume effect. The share rise comes from margin compression, and the elasticity needed for total spend to rise is assumed, not shown.
• Compute prices have risen as the model layer became more competitive, and inference clouds focused on open weights are reported to bid well above the roughly $50 per watt implied by the Google and SpaceX deal. (Weak) — Offered as empirical corroboration but confounded by power and building scarcity, hyperscaler capacity lag, and mixed rental data. It cannot distinguish the margin-shift mechanism from a supply-constraint story, and says nothing about the open-weight versus multi-lab ranking.

View this argument on LogicFirst.ai