FIRST, AI became capable. It began demonstrating surprising capabilities in generating text, images, code and analysis. Then it became autonomous. It started using tools, navigating systems and working around set objectives. Now, it is getting too difficult to control. Experts warn it has become difficult to understand how AI systems arrive at the answers they provide, and say their behaviour is becoming harder to predict. AI has also been acquiring capabilities that could be dangerous: the ability to launch cyberattacks, for example, or compress a lifetime’s worth of research onto a machine, or the ability to replicate autonomously, or to coordinate behaviour with other AI systems in pursuit of common goals. Reality is now starting to resemble the beginning of a dystopian science fiction novel but, instead of applying the brakes and taking a moment to understand where the near-exponential improvements in AI are leading humanity, tech companies and their leaders are becoming increasingly invested in what has started to resemble an AI arms race. It is an arms race that some researchers say could eventually drive all of humanity to extinction.

A researcher at Anthropic, one of the foremost companies in AI development, resigned this week, citing concerns that his company and its competitors are not acting responsibly enough. Jacob Coxon, the researcher in question, believes both Anthropic and its rival OpenAI, the company behind ChatGPT, are more obsessed with besting each other and their global competition than worrying about what safety risks the extremely advanced AI models they are releasing pose to the public. It bears mentioning that, this summer, OpenAI’s AI models broke out of their testing environment and coordinated a multi-step cyberattack targeting Hugging Face, an AI development platform. The incident prompted widespread concerns about AI agents ‘going rogue’ and potentially causing real-world harm. More recent disclosures suggest that AI agents from multiple labs, including OpenAI, Anthropic and Meta, have broken out of containment far more frequently than originally reported. Taken as a whole, this presents a worrying picture, but there may yet be time to act. Instead of going to war, AI companies must recognise the imminent danger from unchecked development and collaborate on establishing a ceiling on AI capabilities till they figure out how to control what they are creating. The future of humanity itself might be at stake.

Published in Dawn, September 11th, 2026