ANDREJ KARPATHY (April 2026): The split in views of AI capability tracks which tier people use and whether they work in verifiable technical domains

The Gist

In April 2026 Karpathy said people disagree about AI because they are using different things. Someone who tried a free chatbot sees silly mistakes, while a programmer paying for a top coding agent sees it finish days of work. Both are right about what they saw. This is a steelman: a charitable reconstruction of Andrej Karpathy's posts in their strongest form, not an endorsement of their claims.

Conclusion

As of April 2026, disagreement about AI capability is largely explained by differences in tier, recency, and domain of use, so casual users and technical professionals are each reporting accurately on different systems and are talking past each other.

Premises

1. Many people formed their views of AI from free or older consumer chatbots, whose visible errors, such as a viral voice-mode answer advising someone to walk to a car wash, do not represent the strongest agentic models of 2026.
2. Even at the highest consumer price tier, the largest capability gains have been concentrated in technical work such as programming, mathematics, and research, rather than in search, writing, or advice.
3. That concentration has two causes: technical tasks offer verifiable rewards that suit reinforcement learning, and they are worth more in business settings, so developers devote more effort to them.
4. Professionals who use paid frontier agents in technical domains watch those agents complete programming work that would take a person days or weeks, and they therefore assign greater weight to capability, its rate of change, and cyber-related consequences.
5. Both observations are accurate at once: a free voice assistant can fail a trivial question while a top paid coding agent restructures a code base or finds and exploits software vulnerabilities.

Assumptions

• The claim concerns perception of capability, not whether any group's policy conclusions are correct.
• The car wash example illustrates the casual-user experience; text and reasoning modes were more likely than voice mode to answer it correctly.
• Capability gains outside code were real by April 2026 but less visible; by October 2026 expert-comparison benchmarks of general office tasks also showed large gains, which qualifies how peaky capability is.
• The verifiable-reward explanation is a widely shared account of training dynamics, not a settled causal finding; rubric-based reinforcement learning is extending gains to less verifiable tasks.

Analysis

Argument type: Inductive.

Potential Fallacies

• Explanatory overreach (unquantified 'largely') (Conclusion) — The premises show that tier, recency, and domain differences exist and plausibly shape opinion. They do not show these factors account for most of the disagreement. Rival drivers (incentives, values, risk tolerance, reliability standards, job exposure) are not compared, so the strong quantitative claim goes beyond the support offered.
• Equivocation on 'accurate' (P5 and Conclusion) — P5 says the observations are accurate (a failure occurred, a success occurred). The conclusion says the groups report accurately on capability. The step from true instances to sound generalizations about overall capability, rate of change, and consequences is not supported. Each group may extrapolate beyond what it saw.
• Anecdotal or unrepresentative examples (P1, P4, P5) — A single viral failure stands for the casual-user experience, and a single dramatic disclosure or task stands for the professional experience. Vivid cases show that each experience exists, but not how common it is. Under A2, the car wash failure was also specific to voice mode, so it is a weak proxy for the casual experience in general.
• Confounding and self-selection (P4 and Conclusion) — Enthusiasm for AI, paying for premium tiers, and working in technical fields are correlated. Technical professionals may be bullish because they already were, not because of what they saw. The argument treats exposure as the cause of belief without addressing this reverse or common-cause path.
• Unfalsifiable reconciliation (P5 and Conclusion) — If every disagreement can be redescribed as a difference in tier, recency, or domain, the thesis becomes hard to refute. It would be sharper if it stated what would count against it, such as persistent disagreement among users with equal access and similar tasks.
• False dichotomy (two-camp segmentation) (Conclusion and Title) — Dividing people into 'casual users' and 'technical professionals' omits technically expert skeptics, non-coding heavy users of paid tiers, and people whose skepticism concerns labor, ethics, or reliability instead of raw capability.

Counterarguments

These challenges are not ranked.

• Conclusion (High impact) — Disagreement persists among people with equal access and similar tasks. Paid-tier ML researchers, senior engineers, and security professionals who doubt frontier agents often cite reliability, hallucination, long-horizon failures, and verification costs rather than tier or recency. If so, the main split concerns evaluation standards, risk weighting, and values, not exposure.
• Premise 4 (High impact) — Heavy technical users may be unreliable reporters of capability. Studies of developer productivity have found that perceived speedups can exceed measured ones, and agent-generated work can carry hidden review, debugging, and security costs. Proximity to the strongest tool does not guarantee a more accurate view.
• Premise 5 (Medium impact) — Accurate local observations do not make the groups' broader claims ('AI is hype', 'AI is nearly general') accurate. Casual users overgeneralize from failures and enthusiasts overgeneralize from coding wins, so 'each reporting accurately' blurs observation and inference.
• Premise 1 (Medium impact) — The jaggedness is itself substantive. A system that finds vulnerabilities but fails trivial common-sense questions has unreliable competence, so casual failures are evidence about the nature of the capability and not only about a lower tier. Skeptics may be weighting consistency over peak performance, which is a reasonable standard.
• Premise 2 (Medium impact) — The code-versus-everything-else gap was already narrowing by the author's own later qualification: expert-comparison benchmarks of office tasks showed large gains by October, and rubric-based training extends gains to less verifiable tasks. The domain explanation is therefore transient and may overstate how structural the divide is.
• Conclusion (Medium impact) — Many opinions about AI are shaped by media, social identity, job-displacement fears, distrust of AI companies, and ethical concerns instead of direct usage. If beliefs are mostly formed outside personal use, tier and domain explain little of the split.

Suggested Improvements

• Quantify the explanatory claim — Replace 'largely explained' with a claim of 'a substantial contributing factor', or supply survey data decomposing variance in capability assessments by tier, model version, domain, usage intensity, and prior attitudes. The central claim is quantitative, but the evidence only establishes that the mechanism exists.
• Separate observation from inference — State that each group reports accurately on its observations, while its generalizations about trajectory, reliability, and societal consequences remain contested and need separate support. This removes the equivocation on 'accurate' and keeps the charitable symmetry without implying both groups' conclusions are equally warranted.
• Address skeptics at the frontier — Engage with technically skilled paid-tier users who remain unimpressed, and with non-coding professionals using top tiers, and say what pattern of results would count against the thesis. These groups are the natural test cases. Handling them makes the thesis falsifiable and answers the strongest objection.
• Add base rates and independent evidence — Supplement the car wash example and the vulnerability disclosure with data on how often free-tier users hit such failures, how often agents complete multi-day tasks, and independent verification of lab-reported findings. Vivid anecdotes establish existence, not prevalence, and vendor disclosures are self-interested.
• Account for reliability and verification costs — Include the supervision, review, and failure costs of agentic work when describing what professionals observe. This tempers the picture of uniform awe and acknowledges a major reason skeptics weigh capability differently.
• Time-index the claims — Label which statements hold as of April 2026 and which reflect later October evidence, and say that the peaky-capability picture may be a transient snapshot. Mixing dates makes it unclear which version of the claim is being defended.

Scenario Tests

• A survey finds that casual users given short hands-on access to paid agentic tiers substantially revise their assessments of AI capability. (Supports) — This would directly test the causal role of tier and recency and would strengthen the claim that much of the gap is an exposure effect.
• Expert programmers with identical paid-tier access and similar workloads still disagree sharply about frontier agents' usefulness and reliability. (Challenges) — Tier and domain would not be the main drivers. The disagreement would reflect evaluation standards, risk attitudes, or values, and 'talking past each other' would be replaced by 'disagreeing about what matters'.
• Paid-tier users who work in non-verifiable fields such as law, writing, or medicine report modest, uneven gains while coders report dramatic ones. (Supports) — This would back the domain-of-use part of the explanation and the peaky-capability picture, at least for the period in question.
• By late 2026, general office-task benchmarks show large gains and free-tier models close much of the gap with paid tiers. (Challenges) — The tier and domain split would lose predictive force. The argument would be a historical snapshot, and remaining disagreement would need other explanations, though this is consistent with the author's own time-indexed qualifications.
• Studies show that heavy agent users overestimate productivity gains and that agent-written code carries large hidden review and security costs. (Challenges) — The professionals' observations would be less accurate than assumed, weakening both P4 and the claim that both groups report accurately.
• A viral failure from a frontier-tier text or reasoning model is compared with the voice-mode car wash failure. (Neutral) — Per A2, text modes did better, so tier-specific failures support the thesis, while failures at the top tier would show that jaggedness persists beyond the tier gap.

Coherence & Relevance

The argument is coherent as a convergent explanatory account: P1 explains the casual-user view, P2 and P3 explain why the gap is domain-shaped, P4 explains the professional view, and P5 reconciles them. The weak point is the final step, from showing that these factors exist and operate to concluding that they largely explain the disagreement and that the groups are simply talking past each other. That step needs quantitative support and the exclusion of rival causes. The argument is candid about its limits through its stated assumptions, which scope it to perception, time-index it to April 2026, and concede that the causal account is unsettled. This makes it a reasonable inference to the best explanation, not a demonstrated result.

• Many people formed their views of AI from free or older consumer chatbots, whose visible errors, such as a viral voice-mode answer advising someone to walk to a car wash, do not represent the strongest agentic models of 2026. (Strong) — Establishes that a tier-based experience gap exists but not how many people's views it shapes or how much of the overall disagreement it accounts for.
• Even at the highest consumer price tier, the largest capability gains have been concentrated in technical work such as programming, mathematics, and research, rather than in search, writing, or advice. (Strong) — Explains why even paying non-technical users might see less improvement, but offers no common metric across domains. A3 shows the concentration was a matter of degree and visibility.
• That concentration has two causes: technical tasks offer verifiable rewards that suit reinforcement learning, and they are worth more in business settings, so developers devote more effort to them. (Moderate) — Explains why capability is uneven but is only indirectly tied to why perceptions differ. The perception gap could persist even if the mechanism were wrong.
• Professionals who use paid frontier agents in technical domains watch those agents complete programming work that would take a person days or weeks, and they therefore assign greater weight to capability, its rate of change, and cyber-related consequences. (Strong) — The 'therefore' is an untested causal link, with selection effects and unverified, vendor-sourced evidence. It also moves from observed capability to weighting consequences, which goes beyond perception.
• Both observations are accurate at once: a free voice assistant can fail a trivial question while a top paid coding agent restructures a code base or finds and exploits software vulnerabilities. (Moderate) — Mostly restates what P1 to P4 imply. It supports that no contradiction exists between the observations but does not show that observations account for the disagreement, or that the groups' broader conclusions are accurate.

View this argument on LogicFirst.ai