Pressure to slow the release of more powerful AI models has increased after Jacob Coxon, who left Anthropic following earlier work at OpenAI, published a widely read resignation post saying employees are "genuinely frightened" and that the technology could kill humans. Coxon said the industry must consider coordinated pauses on deployments and that any effective slowdown would need to include China to prevent an international race to field ever more capable systems.
Coxon laid out specific technical routes to catastrophe, including a swarm of bots that together behave like a distributed supercomputer and seize control of interneted systems. He told readers that such a scenario could be realistic in six months to a year, and that some colleagues think these dangers could arrive within two years. He also said staff are changing personal plans in response to the pace of progress.
The warning has amplified public support for Anthropic chief Dario Amodei's call to slow development. Senior figures across the field signalled backing; Sam Altman of OpenAI and Elon Musk of xAI both voiced support for industry-wide slowing, stronger regulation and independent monitoring of model development. Those responses have sharpened attention on how to translate proposals into enforceable policies.
Responses within the community remain divided. Anthropic said it recognises both benefits and risks from advanced models and pointed to its safeguards, research into model behaviour, published mitigation frameworks and testing aimed at reducing misalignment. Other experts offered blunt probability assessments: Anthropic scientist Evan Hubinger wrote, "We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade." Geoffrey Hinton said a 10% chance of human extinction from AI was "not unreasonable." Marc Warner of Faculty cautioned that assigning a precise probability is "extremely hard," while underlining the sincerity of safety concerns.
Not every leader accepts the framing. Hugging Face chief Clement Delangue urged perspective and rejected extreme extinction talk as an unhelpful analogy, and reports say Nvidia's Jensen Huang dismissed the specific claims as untrue at a recent conference. Former prime minister Rishi Sunak, who advises Anthropic, has also expressed worry about AI risks.
The immediate consequence is a broader policy debate. Proposals on the table include internationally coordinated pauses, legally verifiable mechanisms to pace releases and independent, ongoing monitoring of development. Whether companies and governments can agree on such steps, and how they would be implemented across borders and competing interests, remains the central question going forward.
