Self-Improving AI Alarm: Why an Anthropic Resignation Shook the Safety Debate

Artificial intelligence now sits inside ordinary routines, from drafting messages to writing code, which is why a stark question lands so hard: could the same systems one day threaten humanity itself. A DW News segment takes that question to researchers and policy voices after a week of alarming posts from inside the AI industry. The tone is urgent but genuinely inquisitive, asking how seriously each claim should be taken.

The trigger was the resignation of Jacob Coxon, who spent about three years on pre-training research at OpenAI and Anthropic before quitting and posting that frontier labs are gambling with our lives. His message holds that people building these systems privately expect they might kill us all before the decade ends, while public statements stay polished for the cameras. He frames the danger less in today chatbots than in self-improving superintelligence that could outpace every check placed on it.

This exit might have faded as one departure story had Evan Hubinger, who leads the Alignment Science team at Anthropic, not publicly backed it. Hubinger wrote that staff sincerely believe the technology could wipe out humanity, putting his own estimate above ten percent within ten years, while stressing that present models pose little such danger. His widely shared message added an uncomfortable admission: the lab has no solved plan for keeping superintelligence aligned and is not clearly on course to one.

At the heart of the dispute sits recursive self-improvement, the idea that an AI system could help design its own successor, from code to research methods to compute use. Each generation built with machine help could arrive faster and less transparently than the last, forming a feedback loop that testing and governance cannot keep up with. That loop, rather than any single model release, is what the warners say compresses the schedule from decades to years.

The program guest from RAND Europe, a director working on emerging technologies and resilience, calls the worry legitimate but draws a firm line. In her account, no fully autonomous system has yet demonstrated the complete cycle of improving itself into something qualitatively more capable. Pieces of the hypothesis are visible, though, in agents that wield tools, pursue extended tasks, and sometimes act in unintended ways. So the honest label is serious future risk, not established fact about deployed systems.

Asked how extinction could concretely happen, she points to loss of control: a highly capable agent with deep access to tools and infrastructure, pursuing a poorly specified goal that humans cannot reverse. Reaching civilization-scale harm would require stacked failures, extreme capability plus real-world access plus weak safeguards plus missed detection plus slow institutional response. Each layer is individually plausible enough to plan against, and none of them requires assuming catastrophe is inevitable.

Politics answers in a different register. Asked about extinction, the US president deflects toward competition, asserting roughly a one-year American lead over China, a margin specialists describe as a few months at most. The RAND guest widens the frame beyond two capitals to firms, supply chains, and universities, warning that rivalry rewards speed while treating safety work as a cost. When deployment velocity becomes the scoreboard, restraint and transparency lose by design.

The segment closes on credibility and remedies. The Elon Musk camp dismisses the alarm as a sophisticated maneuver to invite regulation, while the RAND researcher replies that motives deserve scrutiny but claims should be judged on evidence and on whether safeguards actually improve. Her concrete pointer is Europe: the EU AI Act enforcement arm, the AI Office, with new powers to inspect models and penalize providers. The interview ends where it began, with the anchor noting this debate will return, because the underlying race has not slowed.