Dario Amodei: Global pacing should pursue verified Levels 1-4 with China where feasible, protecting democratic lead and using informal norms when formal deals fail

The Gist

Amodei says the world should try deals with China that start with easy shared bans like bioweapons, then testing rules, then maybe speed limits on AI building AI, and only a full pause if verification is rock solid, without ever trusting a deal that would let China race ahead if it cheated. Steelman reconstruction for shared understanding; not an endorsement of Anthropic policy positions.

Conclusion

Global coordination should seek verified agreements with authoritarian states on a ladder from bio-use bans through pre-release testing and RSI speed limits to full pacing, only with ironclad verification or limited scope that protects democratic lead, while informal norms remain valuable if formal deals stall.

Premises

1. Worldwide pacing is desirable in parallel with democratic pacing but much harder, especially regarding cooperation with China.
2. Naive symmetric restraint that China then defects from could yield geopolitical dominance by the defector; agreements need ironclad verifiability or limited scope.
3. Near-term global pacing decisions should protect the lead of the U.S. and allies.
4. Level 1: ban narrow dangerous uses such as AI for biological weapons production.
5. Level 2: mutual pre-release testing for acute cyber, biology, and alignment risks via a global standards body.
6. Level 3: RSI speed limits slowing improvement from extremely fast to only somewhat fast, analogous to SALT.
7. Level 4: full pacing or pause of overall AI development, unlikely soon given defection incentives.
8. Any cooperation extends time for democratic-frontier pacing; aim high while treating lower levels as more realistic.
9. Even without formal agreements, informal norms and information-sharing about RSI and misalignment can reduce reckless racing.

Assumptions

• Level numbering and feasibility rankings are Amodei's stipulated ladder.
• SALT analogy illustrates mutual caps preserving balance, not isomorphic RSI metrics.
• Research residual: U.S.-China dialogue prospects contested; secret military model verification remains the hard problem.

Analysis

Overall strength: Moderate. Argument type: Inductive.

Premise Strength

• Worldwide pacing is desirable but much harder, especially with China (Moderate) — Plausible general geopolitical reasoning, but relies on assumed incentive structures rather than direct evidence of Chinese AI policy or negotiating posture.
• Naive symmetric restraint risks defector dominance; agreements need ironclad verification or limited scope (Moderate) — Sound game-theoretic logic consistent with general defection dynamics in bilateral agreements, but the necessity claim is asserted rather than derived, and creates unresolved tension with the acknowledged verification gap (A3).
• Near-term decisions should protect the lead of the U.S. and allies (Weak) — A contested normative commitment asserted without independent justification; multiple analyses flag this as strategically self-undermining, since it signals asymmetry that a negotiating counterpart would likely reject, and ethically privileges bloc advantage over universal risk reduction.
• Level 1: ban narrow dangerous uses (bio-weapons) (Strong) — The most defensible and independently plausible rung; narrow, already largely illegal uses provide low-hanging-fruit cooperation with minimal verification burden.
• Level 2: mutual pre-release testing via a global standards body (Moderate) — Conceptually reasonable but assumes institutional capacity (a functioning cross-border standards body) that does not currently exist and would require years to build even under cooperative conditions.
• Level 3: RSI speed limits analogous to SALT (Weak) — The SALT analogy is the argument's most vulnerable point: RSI lacks an agreed, observable metric comparable to missile/warhead counts, and the argument's own assumption (A2) concedes the disanalogy without resolving it.
• Level 4: full pacing, unlikely soon given defection incentives (Moderate) — Appropriately hedged and consistent with the argument's own defection logic; an epistemic virtue rather than a weakness, though it functions partly as a rhetorical bookend.
• Cooperation extends pacing time; aim high, treat lower levels as more realistic (Weak) — Largely restates strategic preference already established in P1/P2 without independent evidentiary support or probability estimates for which levels are actually achievable.
• Informal norms and information-sharing can reduce reckless racing absent formal deals (Moderate) — A plausible fallback with some historical precedent (informal restraint norms in other domains), but no mechanism is specified for how unenforceable norms would overcome strong unilateral racing incentives, and the claim is difficult to falsify.

Potential Fallacies

• False analogy (P6, supported by A2) — The comparison to SALT borrows credibility from a historically successful arms-control framework, but SALT involved countable, physically verifiable assets (missiles, warheads) detectable via satellite, whereas RSI speed has no agreed metric and no observable signature. The argument itself (via A2) concedes the analogy is not isomorphic, which mitigates but does not eliminate the risk that readers import unearned confidence from the precedent.
• Question-begging condition (unresolved circularity) (P2, in tension with A3) — The argument's operative requirement is that cooperation proceed only with 'ironclad verification or limited scope,' yet the same framework concedes that verifying secret military AI models is an unsolved 'hard problem' (A3). This makes the central gatekeeping condition for Levels 2-4 dependent on an admittedly unresolved capability, without the argument bridging this gap or explaining what would count as sufficient interim verification.
• Loaded framing / presumptive characterization (P2 ('naive symmetric restraint')) — Labeling full cooperative restraint as 'naive' presupposes that skepticism toward China is the only sophisticated position, foreclosing consideration of good-faith cooperative framings without independent argument for why defection is the more likely outcome than reciprocal restraint.
• Structural self-undermining asymmetry (P2 combined with P3) — The argument requests what is framed as reciprocal, verifiable cooperation (P2) while simultaneously making explicit that the goal is to preserve one side's relative advantage (P3). This built-in asymmetry gives the other party a rational basis to view the entire framework as strategic messaging rather than genuine cooperation, which could undermine negotiability even at the most achievable level (bio-use bans).

Counterarguments

• P2 combined with A3 (High impact) — If ironclad verification of secret military AI models is genuinely unsolved, as the argument itself concedes, then the entire ladder above Level 1 is currently unactionable — the proposal reduces to pursuing unverifiable diplomacy while unilateral racing continues under cover of 'protecting the lead.'
• P3 (High impact) — Framing cooperation as conditional on preserving the U.S./allied lead is likely to be read by China as evidence of bad faith rather than genuine multilateralism, undermining negotiability even at the most achievable tier (bio-use bans) before verification questions are ever reached.
• P6 (Medium impact) — SALT succeeded because missile and warhead counts were physically countable and satellite-verifiable; RSI speed is a software/algorithmic phenomenon with no agreed unit, no physical signature, and rapid reversibility, making the analogy's practical transferability doubtful regardless of its rhetorical appeal.
• Conclusion (Medium impact) — The proposal treats AI governance as fundamentally bilateral (U.S.-China), but frontier capability is diffusing across other states and open-source/non-state actors; an agreement that pacifies only the two largest labs could be undermined by third parties not bound by the ladder, making the framework's risk-reduction claims overstated.
• Conclusion (ethical dimension) (Medium impact) — By subordinating cooperative catastrophic-risk reduction to the preservation of a particular geopolitical power balance, the argument treats the safety of populations outside the 'democratic' bloc as instrumental to bloc-preservation rather than as an end in itself, a value choice that is assumed rather than defended.

Suggested Improvements

• Verification specificity — Replace the assertion that agreements 'need ironclad verifiability' with a concrete research and diplomatic roadmap (e.g., compute governance, hardware attestation, third-party auditing) specifying what would need to be true for verification to become tractable. This would close the gap between the requirement in P2 and the admitted unsolved status in A3, converting an unresolved precondition into an actionable research agenda.
• Negotiating posture / framing symmetry — Reframe the goal from 'protect democratic lead' to 'mutual catastrophic-risk reduction, with lead-preservation as one legitimate but secondary interest,' presented symmetrically. Reduces the risk that the counterpart perceives the entire framework as strategic messaging, which several analyses identify as a primary threat to even the most achievable cooperation tier.
• Empirical grounding — Incorporate historical base rates on compliance/defection in comparable dual-use treaties (nuclear NPT, chemical weapons conventions, biological weapons conventions) to calibrate feasibility claims for each ladder level. Would replace assumed feasibility gradations with evidence-based probability estimates, addressing the concern that the ladder implies more empirical precision than currently exists.
• Scope of actors considered — Extend the framework beyond a strictly bilateral U.S.-China frame to address other state actors, open-source diffusion, and multilateral institutions. A bilateral agreement could be undermined by third parties not party to it; addressing this widens the framework's real-world robustness.
• Source transparency — Explicitly acknowledge the author's institutional position as a frontier AI lab CEO and the potential for the proposed framework to serve competitive as well as safety interests. Improves audience calibration and pre-empts a natural conflict-of-interest critique that several analyses identify as a live concern in AI policy discourse.

Scenario Tests

• Verification technology for secret military AI training remains unsolved indefinitely (Challenges) — Levels 2-4 become permanently inert, leaving only symbolic Level 1 bans achievable — a much weaker outcome than the ladder implies, and the argument's own admission (A3) already anticipates this possibility without resolving it.
• China's negotiators explicitly reject the framework upon recognizing the 'protect democratic lead' condition embedded in P3 (Challenges) — Even Level 1 (the most defensible rung) could fail to gain traction, since the perceived asymmetry undermines trust before technical verification questions are reached.
• Chinese frontier models achieve rough capability parity with U.S./allied models (as suggested by post-2023 developments) (Challenges) — Undermines the premise that a 'democratic lead' currently exists and is protectable, weakening the strategic rationale embedded in P3 and the conclusion's asymmetric verification requirement.
• A narrow, symbolic bio-weapons-use ban is successfully agreed as a confidence-building measure (Supports) — Validates the ladder's incremental logic at its most defensible level and could plausibly build toward higher-trust engagement, supporting P8's optimism about extending pacing time.
• Track-two dialogues and informal researcher exchanges continue even as formal treaty talks stall (Supports) — Provides modest support for P9's fallback claim, though the effect size and durability of such norms under deteriorating relations (e.g., over Taiwan or export controls) remain uncertain.

Coherence & Relevance

The argument is internally coherent as a cumulative policy case: each premise maps onto a distinct clause of the conclusion (pursue the ladder; require verification or limited scope; protect the lead; fall back on informal norms), and no formal logical fallacies (e.g., affirming the consequent, undistributed middle) are present. However, coherence at the level of structure does not resolve two load-bearing gaps that recur throughout: the unresolved tension between the verification requirement (P2) and the admitted verification difficulty (A3), and the potential self-undermining effect of pursuing ostensibly symmetric cooperation while explicitly prioritizing asymmetric lead-protection (P3). These gaps mean the argument functions more persuasively as a structured way of thinking about graduated cooperation under uncertainty than as a validated case that the proposed ladder will succeed in practice.

• Worldwide pacing is desirable but much harder, especially with China (Strong) — Establishes the motivating problem but does not itself justify the specific ladder structure that follows.
• Naive symmetric restraint risks defector dominance; needs ironclad verification or limited scope (Strong) — Functions as the conclusion's central necessary condition, but the 'ironclad' standard is asserted rather than shown to be achievable, leaving a load-bearing gap between this premise and A3.
• Protect the lead of the U.S. and allies (Strong) — Directly shapes the conclusion's conditional structure, but its normative status is assumed rather than argued, and it creates tension with the cooperative aims in P1/P8.
• Level 1-4 ladder (P4-P7) (Strong) — These are stipulated categories (per A1) that structure the conclusion's content, but their ordinal feasibility ranking is asserted rather than empirically derived.
• Cooperation extends pacing time; aim high, treat lower levels as realistic (Moderate) — Largely restates the strategic logic of P1/P2 rather than adding independent support; functions as connective tissue rather than new evidence.
• Informal norms can reduce reckless racing absent formal deals (Moderate) — Supports the conclusion's fallback clause but lacks a specified mechanism, making it somewhat disconnected from the more rigorously argued verification-based premises.

View this argument on LogicFirst.ai