SABINE HOSSENFELDER: Expert AI forecasts err in a predictable direction, too slow on software capability and too fast on physical change, so judge AI by friction

The Gist

Sabine Hossenfelder argues that experts keep getting AI wrong in a predictable way. They expect too little from AI in math and coding, and too much, too soon, in jobs and the physical world. So don't take their median forecasts at face value; ask how much building and red tape a change needs. The forecasters' own review agrees on the first half but recommends better forecasts, not ignoring experts. This packet steelmans the argument Sabine Hossenfelder makes in this video, in its best light, for analysis; it is not an endorsement of her conclusions.

Conclusion

Median expert forecasts of AI have proved unreliable in a consistent pattern: they underestimated capability in software domains such as mathematics and coding, often by years, and overestimated how quickly AI would change jobs and physical work. They should therefore not be taken at face value; a better guide is how much physical build-out and institutional change a domain requires, which predicts fast progress in software-only work and slow change elsewhere.

Premises

1. Expert panels surveyed in 2025 and 2026 have already underestimated AI progress in mathematics and coding, in one case by a factor of nearly four and in another, on a Millennium Prize Problem, within about a year if the claimed proof holds, so median expert forecasts of near-term AI capability in these domains should be treated as likely to run low.
2. Underestimation of AI capability is a recurring pattern in forecasts made since 2022, across different expert groups and fields, and is not explained by any single surprise such as the release of ChatGPT; it is a systematic tendency that users of such forecasts should correct for.
3. Because AI now makes plausible research papers cheap to produce in large numbers, the binding question for scientific publishing has shifted from whether AI-written work can pass as publishable to who is accountable for what it says, and platforms are already responding with limits and author-responsibility rules.
4. A plausible, though unproven, contributor to the underestimates is that forecasters anchor on the slower pace of academic research and underweight how much hundreds of billions of dollars in private investment can accelerate AI development.
5. Expert error about AI has not run in only one direction: forecasts that AI and automation would rapidly take over jobs and physical work, such as truck driving, have run ahead of what has happened, so the same experts who underestimated capability have overestimated deployment in some domains.
6. Leopold Aschenbrenner's forecast that AGI is plausible by 2027, with an intelligence explosion to follow, is the clearest example of overestimating AI's pace, because the scale he projected depends on power plants and data centers that take years to build; that investors committed tens of billions of dollars to his views shows how much weight such forecasts carry.
7. The pace at which AI changes a domain depends mainly on how much physical change and institutional approval that change requires: it is fast in mathematics, coding, and publishing, where output is digital, and slow in building labs, vehicles, robots, power plants, mines, and company workflows, where change is likely to take decades.
8. Near-term progress from AI will be fastest in work that software alone can do, including theoretical problems in physics and related fields that may be solvable by combining techniques already in the literature, and because AI tools are widely available, researchers outside AI companies can contribute to that progress.

Assumptions

• The forecasting group's remedy is to discount median forecasts and weigh more accurate subgroups and model-based forecasts, not to disregard expert input, and some expert forecasts (data center capacity, electricity share, public opinion of AI) have tracked reality closely.
• The forecasting group's own review warns that interim accuracy checks are biased toward catching underestimates, because a forecast can be shown too low as soon as reality passes it but shown too high only after its deadline, and most of its AI questions are unresolved.
• 'Don't listen to the experts' is read as advice not to defer to median expert forecasts, not as advice to ignore expertise.
• As of October 2026, most long-term forecasts in the surveys cited have not resolved.

Analysis

Argument type: Inductive.

Potential Fallacies

• Hasty generalization / small sample (P1, P2, P5, Conclusion) — A few salient cases (a factor-of-four miss, a conditional Millennium Prize claim, truck driving, one AGI forecast) are used to establish a 'systematic' and 'predictable' directional pattern. Without a defined reference class or a full set of resolved forecasts, the cases show that errors occurred, not that they have a consistent direction.
• Asymmetric resolution / selection bias (P1, P2 (against the stated assumptions on interim checks and unresolved questions)) — Underestimates become visible as soon as reality passes a forecast, while overestimates only show after the deadline. Early checks therefore skew toward underestimation even if forecasts are unbiased. The stated assumptions acknowledge this, but the conclusion still speaks of a proven, consistent pattern. A provisional reading would fit the evidence better.
• Premature classification of an unresolved forecast (P6) — Aschenbrenner's 2027 forecast has not reached its deadline, so calling it the 'clearest example of overestimating' is a judgment about plausibility, not an observed error. This applies the interim-check logic selectively, since overestimates can only be confirmed after their deadlines. He is also an individual forecaster, not a median panel.
• Post hoc theorizing / unfalsifiable flexibility (P7, Conclusion) — The friction heuristic was fitted to outcomes already known and has no stated friction measure or out-of-sample test. Without a pre-specified measure, fast outcomes can be attributed to low friction and slow outcomes to high friction, so almost any result can be accommodated.
• Equivocation on 'the experts' (P5, Conclusion) — Capability-survey panels (AI researchers, forecasters) are merged with the sources of job-automation and truck-driving predictions (executives, vendors, consultancies, economists). Treating them as one group that errs in both directions is asserted rather than shown, and the two-way pattern may belong to the broader forecasting ecosystem.
• Conditional claim used as evidence (P1) — The Millennium Prize example holds only 'if the claimed proof holds.' An unverified result cannot yet anchor a claim of forecast error, and if verification fails this illustration drops out.
• Investment as a signal of accuracy (P6) — Capital committed to a forecast shows its influence, not its correctness. The scale is also loosely stated and may blend a specific fund with industry-wide capex driven by many other factors.
• Partial circularity (P2) — P2 asserts that underestimation is systematic and recurring, which restates the first half of the conclusion rather than supplying independent support. The groups it cites also share information environments and exposure to the same surprises, so their misses are not fully independent evidence.
• Digital/physical dichotomy (P7, P8) — Domains are sorted into fast 'software-only' and slow 'physical or institutional' categories. This hides hybrid cases (drug discovery, chip design, regulated software) and the coupling between the two: AI software progress itself depends on physical compute and power. P3's accountability rules show publishing also faces institutional friction.

Counterarguments

These challenges are not ranked.

• Premise 2 and Conclusion (High impact) — Because underestimates surface early and overestimates only after deadlines, and most questions are unresolved, the observed 'direction' may be an artifact of when the check was run. Resolved cases cluster where AI happened to surprise on the upside. Ordinary forecasting noise in a fast-moving field, plus a short boom-era window, could produce the same record without any stable bias. Earlier AI eras were marked by overoptimism.
• Premise 7 (High impact) — Fast progress in math and code coincides with cheap automatic verification (proof checkers, unit tests), abundant data, and reinforcement learning on checkable rewards. This rival explanation predicts the same pattern but gives different answers for digital domains that are hard to verify, such as biology, law, and medicine. 'Software-only' does not guarantee speed.
• Premise 5 (Medium impact) — Truck-driving and job-loss predictions came largely from executives, vendors, consultancies and task-exposure studies, not from the AI-researcher and forecaster panels that underestimated math and coding. There is no evidence the same people erred in both directions, so a single 'expert bias' is not demonstrated.
• Premise 7 (Medium impact) — Friction is not static. Software gains (AI-designed chips, robotics control, lab automation, construction planning) and large capital flows may erode physical and institutional friction faster than a 'decades' extrapolation assumes. Robotaxi expansion, data-center build-out, and rapid manufacturing scale-up in some jurisdictions already show variation.
• Premise 6 (Medium impact) — Aschenbrenner's forecast has not resolved, and if capability keeps rising and compute and power build-out accelerate, it could prove closer to right than the argument allows. Labeling it wrong now treats an open forecast as a closed one and applies the interim-check asymmetry selectively.
• Premise 1 (Medium impact) — If the claimed Millennium Prize proof fails verification, or the 'factor of nearly four' traces to ambiguous question wording or benchmark redefinition, the headline illustrations of underestimation weaken considerably. Earlier AI-math claims have sometimes been walked back as literature retrieval.
• Conclusion (Low impact) — Software-fast, physical-slow does not mean low impact. Rapid disruption of digital white-collar work, cybersecurity and dual-use risk, and slow-but-large physical change all carry consequences. Reading 'friction' as reassurance could breed complacency about labor transitions and safety preparation.

Suggested Improvements

• Evidence base for the directional pattern — Replace illustrative cases with a pre-specified set of resolved forecasts. Score signed errors with proper scoring rules, separate capability from deployment questions, stratify by forecaster type, and treat unresolved questions as censored data instead of counting only early misses. This would address the selection and resolution-timing problem and show whether the pattern is systematic or an artifact of early checks.
• Testing the friction heuristic — Define friction in advance with measurable indicators (capital intensity, approval times, build-out requirements, verification cost) and test whether it predicts adoption speed on held-out domains better than expert medians or a verifiability-based model. A pre-registered, comparative test would turn a post hoc story into a falsifiable predictor and separate it from rival explanations.
• Treatment of unresolved forecasts — Reclassify Aschenbrenner's forecast and the Millennium Prize claim as open or conditional, and state what outcomes by what dates would count as confirming or refuting each. This applies the argument's own caution about interim checks consistently in both directions.
• Distinguishing forecaster populations — Identify who made the capability forecasts and who made the deployment forecasts, and either show that the same people erred both ways or recast the claim as a pattern across the forecasting ecosystem. It removes the equivocation in 'the same experts' and makes the two-direction claim defensible.
• Calibrating the friction timeline and coupling — Replace 'decades' with domain-specific ranges and uncertainty, and model feedback between software capability and physical build-out, including the dependence of AI progress on compute and power. It holds the author's own forecasts to the standard she applies to experts and handles the coupled, changing nature of the system.
• Tightening the stated conclusion — State the conclusion provisionally: median forecasts in software-verifiable domains have so far tended to run low in early-resolving checks, and deployment forecasts have tended to run high, with a recommended correction of unspecified size pending resolution. This matches the confidence of the claim to what the evidence and the stated assumptions support.

Scenario Tests

• The claimed Millennium Prize proof is independently verified, and further AI results in mathematics and coding continue to beat expert median timelines. (Supports) — This would strengthen the software-underestimation claim and the case for discounting medians in verifiable domains, though it would not by itself confirm friction as the cause.
• The claimed proof fails verification, or the 'factor of nearly four' is traced to ambiguous question wording. (Challenges) — P1 would lose its main illustrations, leaving the underestimation story dependent on a smaller, noisier set of near-term checks.
• Capability progress stalls in 2027-2030 (data limits, scaling returns, reliability), so long-horizon medians prove accurate or too optimistic even in software. (Challenges) — The 'forecasts run low' correction would push users toward overestimating, and the apparent systematic direction would look like a short-window artifact.
• Aschenbrenner-style timelines are largely missed by 2027-2028 because power and data-center build-out lag. (Supports) — This would confirm the physical-constraint reasoning in P6 and the friction thesis, though a single individual's miss says little about median expert forecasts.
• Software gains accelerate AI-designed chips, lab automation and construction planning, so physical build-out outpaces 'decades' estimates in some sectors. (Challenges) — The slow-physical half of the thesis, supported mainly by anecdote, would fail, and the digital/physical divide would look like a coupled continuum.
• Resolved long-term forecasts later show experts were roughly calibrated, apart from a few near-term math and coding misses, with data-center capacity and electricity forecasts tracking closely. (Neutral) — This would support the narrower reading (discount medians, weigh good subgroups) while undercutting the claim of a broad, predictable directional bias.
• Software-only deployments in law, medicine or enterprise coding are held back by liability, verification, and workflow integration for years. (Challenges) — 'Digital' would not guarantee fast change, showing that institutional friction applies inside software domains and that friction is not cleanly separated from software-versus-physical.

Coherence & Relevance

The argument is coherent as a two-part narrative: a descriptive claim that expert errors have a domain-dependent direction, and a prescriptive heuristic (friction) that reconciles the two directions. The stated assumptions successfully narrow the practical advice to 'discount medians, weigh better subgroups, keep expertise,' which is defensible. Coherence weakens at three joints: from a few cases to a 'systematic' pattern (given the acknowledged asymmetric resolution), from consistency with two examples to the claim that friction is the better predictor, and in the internal tension of calling an unresolved forecast an error while warning against premature scoring. P3 and parts of P8 are loosely connected to the main thesis. The argument is best read as a reasonable, provisional heuristic, not a demonstrated law.

• Expert panels in 2025-2026 underestimated AI progress in mathematics and coding (factor of nearly four; Millennium Prize claim). (Strong) — Directly supports the software-underestimation half of the conclusion, but the sample is small, one example is conditional, and measurement criteria are unspecified.
• Underestimation is a recurring, systematic pattern since 2022 across groups and fields. (Strong) — Partly restates the conclusion. It relies on correlated observations and, per the stated assumptions, a measurement process biased toward catching underestimates.
• Cheap AI-written papers shift the publishing issue to accountability. (Weak) — Not about forecast reliability. It serves only as an illustration of a fast digital domain, and its focus on institutional rules sits uneasily with treating publishing as frictionless.
• Forecasters may anchor on academic pace and underweight private investment. (Moderate) — A hedged causal hypothesis that explains the underestimates but is untested and offered with no comparison to alternative explanations.
• Experts have also overestimated deployment (e.g., truck driving), so the same experts erred both ways. (Strong) — Essential for the two-direction thesis, but the 'same experts' identity is asserted, and the examples come from different eras and forecaster pools.
• Aschenbrenner's AGI-by-2027 forecast is the clearest overestimate; investors committed tens of billions. (Moderate) — Unresolved, an individual rather than a median forecaster, and the investment point shows influence rather than accuracy.
• AI pace depends mainly on physical change and institutional approval: fast for digital output, slow for physical build-out. (Strong) — Central to the prescriptive conclusion, but post hoc, unquantified, and not compared with rival explanations such as verifiability. It treats software and physical domains as independent when they are coupled.
• Near-term progress will be fastest in software-only work, including theoretical physics, and outsiders can contribute. (Moderate) — A forward prediction derived from the friction thesis rather than independent support. The physics extension reflects a field-specific assumption and ignores experimental validation bottlenecks.

View this argument on LogicFirst.ai