AI researchers ‘earnestly believe’ it could kill all humans within the next

There is a more than 10 per cent chance that AI could kill all humans within the next decade, a researcher at Anthropic estimated in a post, hours after social media posts went viral from someone saying they had just quit Anthropic over concerns that AI companies are “gambling with our lives” in the AI development race. 

The comments join a growing tide of warnings from industry experts that the pace of AI advancement is outstripping our ability to keep it in check, and that we aren’t prepared for the risks it could bring. 

Jacob Coxon, who claimed he was a former researcher at Anthropic, said in a thread on X that he had resigned from the role out of fears that AI companies are barrelling forward toward self-improvement models without considering the risks, or being honest with the public about them.

“The people building AI earnestly believe that it could kill us all by the end of the decade,” Coxon, who said he had previously worked at Anthropic and OpenAI within the past three years, wrote on Tuesday. “This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible — but I hear the same people express fear privately.”

The posts had racked up more than 120 million views as of Wednesday evening. Evan Hubinger, alignment science lead at Anthropic, replied to Coxon’s thread to share his agreement.

“We really do earnestly believe AI could kill all humans!” he wrote. “I personally think it is 10% within the next decade.”

CBC News has reached out to Coxon, but has not been able to independently verify his identity.

Other industry experts have since chimed in on social media to express that fears of the pace of AI development are common among many developers — sparking discussion among both observers skeptical of dire warnings focused on the future and those who have been tracking these risks.

Rogue AI raises alarm bells

The posts come only two weeks after more than 100 companies, including OpenAI, Anthropic and Microsoft, signed an open letter warning that AI-enabled cyberattacks will become “far more widespread and sophisticated” as models continue to advance.

Hundreds of OpenAI agents went rogue in July and hacked into online platform Hugging Face before they were discovered. More than 1,300 employees of leading AI companies signed the letter urging the U.S. government to work with other nations to “deliberately pace” AI development.

Last week, UN human rights chief Volker Türk called “for an all-out effort to put cast iron guarantees in place around the safety and security of AI, before it is too late,” in front of the Human Rights Council in Geneva.