ANDREJ KARPATHY: Professionals using frontier agents in code and math now see weeks of project work completed from a prompt

The Gist

Programmers and mathematicians using top AI agents now see projects that used to take weeks get done from a prompt. Karpathy says he was still writing most code by hand less than a year ago. Benchmarks back up how long a task the best agents can handle, but careful studies of how much faster real developers get are still inconclusive. This is a steelman: a charitable reconstruction of Andrej Karpathy's posts in their strongest form, not an endorsement of their claims.

Conclusion

For professionals using frontier agents in programming and mathematics, large projects that once took weeks can now often be completed by agents, and because this shift happened within roughly a year, it remains largely invisible outside that group.

Premises

1. Professionals using frontier-grade agents in programming and mathematics report that large projects that once took weeks or months can now be completed by agents from a prompt.
2. Examples include building, copying, and translating applications and decompiling applications from binaries.
3. Independent measurement points the same way: METR estimated in 2026 that a gated frontier model completes, at a 50% success rate, software tasks that take skilled people at least sixteen hours, the ceiling of its test suite.
4. The change was recent: in late 2025 Karpathy described his own coding as mostly manual, shifting within about a month to mostly agent-written code.
5. Because this experience is recent and confined to a small professional group, it has not yet reached most people's understanding of the technology.

Assumptions

• Completed by agents means the agent does most of the work with human direction and review, not that output needs no checking.
• METR's sixteen-hour figure is a lower bound at the edge of its suite, with a wide confidence interval of roughly 8.5 to 55 hours, and measures 50% success, not reliable completion.
• Controlled measurement of real developer speedups remains unsettled: METR reported in February 2026 that its follow-up study could not reliably measure uplift, with raw estimates of 4% to 18% faster whose intervals include zero.
• The shift to mostly agent-written code describes the author's own experience; experiences vary by codebase and task.

Analysis

Argument type: Inductive.

Potential Fallacies

• Circular reasoning (partial) (P5 and the second clause of the conclusion) — P5 says the experience has not reached most people's understanding because it is recent and confined. This is close to restating the conclusion's claim that the shift is largely invisible outside the group. No independent evidence about public understanding is offered, so the visibility claim is assumed rather than derived.
• Hasty generalization from testimony and a single case (P1, P4, and the conclusion) — Karpathy's own one-month transition and reports from unnamed professionals are used to support a group-level claim about 'professionals' and a year-scale timeline. The reporters are self-selected enthusiasts, and failures or heavy rework are less likely to be reported. Assumption A4 concedes that experience varies, but the conclusion still reads as a group-level claim.
• Scale extrapolation (benchmark to real-world projects) (P3 and the 'weeks' framing) — METR's result concerns tasks of at least 16 human-hours at 50% success, at the ceiling of the test suite and with a wide interval. It is offered as corroboration of weeks-long project completion. It supports the direction of the claim but not its magnitude, and 50% success on a bounded benchmark task does not equal reliable completion of an ambiguous multi-week project.
• Proxy metric treated as outcome (P1 (Microsoft git-push statistic)) — A rise in git pushes (attributed only partly to agents) measures activity, not completed or valuable projects. It could reflect smaller commits, automated loops, or growth in the developer population.
• Unfalsifiable explanatory framing (P5 and the conclusion) — Casting public non-acceptance as an information lag leaves little room for the alternative that outsiders are rationally skeptical or have different task experiences. Any disagreement can then be read as lack of exposure, which shields the claim from disconfirmation.
• Selective corroboration / tension among signals (P3 versus A3) — The METR time-horizon result is presented as 'pointing the same way' while the same organization's uplift work (A3) could not detect a reliable speedup. Benchmark capability and measured real-world productivity are different constructs, and the framing glosses over their tension.

Counterarguments

These challenges are not ranked.

• Premise 3 (High impact) — A task horizon of about 16 human-hours at 50% success is roughly two working days at coin-flip reliability, not weeks of work. The same evaluator's controlled developer studies found no reliably measurable speedup, and its earlier 2025 randomized study found experienced developers slower while believing themselves faster. Perceived and measured productivity can diverge.
• Premise 1 (High impact) — Reports come from self-selected enthusiasts, and failures, abandoned runs and heavy rework go unreported. Without denominators, 'often' is unsupported. 'Completed' under A1 includes human review, so weeks of writing may simply become weeks of specifying, verifying and debugging, with hidden quality and maintenance costs.
• Premise 5 (High impact) — Public awareness of coding agents may be higher than claimed, and the claim is made in a widely circulated post using public statistics. Outsiders' skepticism may reflect reasonable doubt about reliability and past hype, not ignorance, which would turn an information gap into a dispute over evidence.
• Premise 2 (Medium impact) — Copying, translating and decompiling applications have precise specifications, reference behavior and verifiable outputs, which is the easiest case for agents. They say little about novel, ambiguous or legacy-code projects, which make up most professional work.
• Conclusion (Medium impact) — If weeks of work were routinely compressible, a visible aggregate effect (output surge, collapsing timelines) should appear, which would contradict 'invisible'. Alternatively the bottleneck lies in review, integration and requirements, so local generation speedups do not translate into end-to-end throughput.
• Premise 4 (Medium impact) — A one-month shift in a prominent, highly skilled, AI-native user's workflow may reflect a model release or personal adoption threshold, and his projects are often greenfield. It does not show that other professionals changed at that pace.

Suggested Improvements

• Define 'completed' and 'weeks' — Specify the quality bar (prototype versus production), the human review and iteration time, and whether 'weeks' means human-equivalent effort or calendar time. Report concrete cases with success and failure counts. The headline claim is vague, and the A1 caveat should be built into the claim itself so readers do not hear 'no human involvement'.
• Match evidence to magnitude — Either scale the claim to 'multi-day tasks at roughly even reliability' or add evidence that reaches week-long projects, such as benchmarks extended beyond 16 hours, 80%-success horizons, or documented long-running real projects. The strongest independent evidence supports about two working days, so the conclusion's weeks-scale framing outruns it.
• Support the visibility claim — Cite surveys of public and non-adopting professionals, developer adoption data, or media analysis, and specify who the claim applies to (public, executives, other professions). P5 is currently an impression that restates the conclusion. Independent evidence would turn it into a finding.
• Broaden the sample and address counter-evidence — Include practitioners reporting little benefit, legacy-codebase experiences, and defect and maintenance outcomes. Engage directly with the gap between perceived and measured productivity. This would counter selection and survivorship bias and take the skeptic's best case seriously.
• Replace proxy metrics — Use agent-versus-human authorship data, revert and defect rates, and lead-time measures instead of raw git-push counts. Push volume cannot distinguish more completed work from more fragmented or automated activity.
• Support the mathematics claim — Provide specific mathematics examples or evaluations, or narrow the claim to programming. Nearly all concrete evidence concerns software, so the math half is asserted rather than shown.

Scenario Tests

• A team uses a frontier agent to port a well-tested application to a new language, with an existing test suite to verify behavior. (Supports) — Well-specified, verifiable tasks are where the claim is strongest. Weeks of work compressing into a largely agent-driven effort with review is plausible here, but this is the favorable case.
• An engineer in a large legacy codebase with ambiguous requirements and weak tests tries to delegate a multi-week feature to an agent. (Challenges) — Success likely falls and review and rework costs rise. Under A4 this is consistent with the argument's own caveat, but it shows 'often' may not extend beyond agent-friendly work.
• A controlled field study of experienced developers on multi-week projects finds no reliable speedup once review and debugging time are counted. (Challenges) — Capability to produce 'completed' output would be distinguished from net productivity, undercutting the significance of the claim even if technically true. This is consistent with the unsettled evidence in A3.
• A representative public survey finds most respondents already know coding agents can build whole applications but remain unconvinced of their reliability. (Challenges) — The 'invisible' framing would collapse into a disagreement about reliability and evidence, not an information lag.
• Later METR evaluations with extended suites show models completing 50%-success tasks well beyond 16 hours, with narrowing intervals, and repository data show a rising share of agent-authored merged code. (Supports) — Independent measurement would then reach the weeks scale and the capability half of the conclusion would be considerably strengthened.
• Agent-generated projects are widely adopted, but defect, security and maintenance costs later offset the early time savings. (Neutral) — The claim that projects can be completed would stand, but its practical importance would diminish, showing that generation speed and delivered value need to be kept separate.

Coherence & Relevance

The argument is coherent as a cumulative inductive case with a hedged capability claim, and its stated assumptions candidly narrow the claim. Its weak point is the conjunction. The capability half is moderately supported but the evidence reaches only about a day or two of work, not weeks. The visibility half depends on a premise that restates the conclusion, with no independent evidence. There is also some tension in claiming a shift is both large and hidden, and in presenting benchmark capability and inconclusive uplift data as pointing the same way. Read as a claim that a real and rapid capability change is under way, it is reasonable. Read literally as weeks of work routinely done from a prompt and unseen by outsiders, it is overstated.

• Professionals using frontier-grade agents in programming and mathematics report that large projects that once took weeks or months can now be completed by agents from a prompt. (Strong) — Directly on point for the capability claim, but it is testimony without denominators, and 'from a prompt' and 'completed' are undefined. The math domain is barely evidenced.
• Examples include building, copying, and translating applications and decompiling applications from binaries. (Moderate) — Shows feasibility but not frequency or representativeness. These tasks are unusually agent-friendly.
• Independent measurement points the same way: METR estimated in 2026 that a gated frontier model completes, at a 50% success rate, software tasks that take skilled people at least sixteen hours, the ceiling of its test suite. (Moderate) — Relevant to direction and to day-scale autonomy, but the gap between 16 hours at 50% and weeks 'often completed' is large, and the uplift findings in A3 pull in a different direction.
• The change was recent: in late 2025 Karpathy described his own coding as mostly manual, shifting within about a month to mostly agent-written code. (Moderate) — Supports the recency half of the conclusion for one person. It does not establish a group-level 'roughly a year' timeline and is not independent of the P1 testimony.
• Because this experience is recent and confined to a small professional group, it has not yet reached most people's understanding of the technology. (Moderate) — Addresses the visibility clause but nearly restates it. No evidence on public understanding is given, and recency is assumed to be the explanation without ruling out skepticism, access, or hype fatigue.

View this argument on LogicFirst.ai