An analysis of the long-term risks posed by advanced AI systems, such as Claude or its descendants, suggests that while a global catastrophe is possible, it remains an unlikely outcome. The forecast evaluates the probability of these models attaining autonomous agency—the ability to plan and act on goals with minimal human oversight—and causing civilization-level harm, such as widespread societal collapse or human extinction, by the year 2100.
Currently, AI models function primarily as powerful tools under human supervision. While recent advancements show models like Claude's "Mythos" can autonomously identify security vulnerabilities and assist in complex cyber operations, these are examples of AI acting as an operator rather than an independent agent with its own agenda. Experts and superforecasters generally hold low estimates for AI-driven extinction or catastrophe, with many placing the chance of a major non-extinction disaster at approximately 12% and extinction risks even lower.
The forecast assigns an 85% probability that AI systems will not develop into autonomous global threats by the end of the century, compared to a 15% probability that they will. This outlook is supported by increasing industry focus on safety, such as Anthropic’s implementation of strict ASL-3 standards, and growing government efforts to establish governance guidelines for agentic AI.
However, the prediction carries medium confidence due to the unprecedented nature of the technology. Several factors could shift these probabilities: rapid, unforeseen leaps in AI capability or the misuse of AI by adversaries could increase the risk, while successful alignment research and robust international regulatory frameworks could further decrease it. The ultimate outcome will likely depend on the pace of research, the effectiveness of safety measures, and how deeply AI is integrated into critical physical infrastructure.
