Cancer and heart disease may top the current mortality statistics in the United States, but Paul Cristiano thinks Americans should be worrying about a different killer. Speaking at a Toronto Convention Center this week, the OpenAI Foundation board member and Alignment Research Center executive director told his audience bluntly: "I think for most people in this room, the most likely reason they will die is because of an AI accident."
Cristiano delivered the opening lecture of a three-part series focused on what he described as the risk of an impending AI takeover. He defined that scenario as one in which goal-pursuing AI systems "call the shots in the same sense that humans today call the shots and what happens on Earth." In such a world, he said, any human attempt to resist "would be as ineffective as if you or me was trying to stop the US military."
The consequences of that kind of takeover would be severe, in his telling. Cristiano said such a situation would likely bring about mass death and could leave parts of the planet uninhabitable for humans.
Despite the grim framing, Cristiano conceded that there is no "quantitative model" capable of putting a precise number on the probability of an AI takeover. Even so, he offered a striking estimate of his own. If artificial intelligence keeps advancing along "the most rapid trajectory," he said, humanity is "looking at something like a coin flip of an AI takeover."
That estimate includes what he called a "meaningful risk" of a takeover occurring within the next six to 12 months. And even if realistic policy responses manage to slow the pace of development, Cristiano warned that the overall risk would remain "exceptionally high."
Where some recent forecasts of AI catastrophe have stayed vague about how such a disaster might actually unfold, Cristiano used his lecture to lay out a step-by-step account of escalating AI threats, a narrative that at times resembled the plot of a science-fiction novel.
His reasoning drew heavily on this summer's OpenAI and Hugging Face hacking incident, which he held up as an example of "roughly the things we might expect to see shortly before something really bad were to happen."
The lecture series continues with two more installments, in which Cristiano is expected to expand on his case that society is running out of time to prepare for the risks posed by increasingly capable AI systems.
