The atheist billionaire who told the Pope AI can't self-govern

Last week, Pope Leo XIV delivered his first encyclical on AI, and sitting beside him was an unlikely ally: a self-declared atheist billionaire who runs one of the most valuable AI companies on earth. Chris Olah, cofounder and interpretability research lead at Anthropic, used the Vatican stage to deliver a message that many in Silicon Valley would rather not hear: AI companies cannot be trusted to police themselves. "Some might believe that matters of AI are best handled by computer scientists like myself," Olah said in his prepared remarks. "They are mistaken." It was a striking admission from a man whose company just confidentially filed for an IPO and is valued at $965 billion, according to the source. Olah's net worth now sits at just under $8 billion, per the Bloomberg Billionaires Index. But his journey to that papal seat was anything but conventional.

Olah grew up in Toronto as a devout evangelical Christian before becoming an atheist at 15. He dropped out of the University of Toronto after a year, then won a $100,000 Thiel Fellowship in 2012 - a program created by Peter Thiel to fund young people who skip college. That fellowship launched him into a career that would eventually make him one of the most influential researchers in AI safety. He spent three years at Google Brain, where he helped pioneer "mechanistic interpretability" - the effort to reverse-engineer neural networks to understand what they're actually doing. At the time, most researchers were focused on making AI more powerful, not on understanding its inner workings. Olah's work, including a paper called "The Building Blocks of Interpretability," offered one of the first windows into how neural networks deduce complex concepts from simpler building blocks. It was niche work, but it caught the attention of OpenAI.

From 2018 to 2020, Olah led OpenAI's interpretability team, where he worked on two landmark projects. The Circuits project aimed to prove that neural networks contain identifiable, human-readable information formed by structured patterns of neurons. The second project discovered multimodal neurons in OpenAI's CLIP model - neurons that would "fire" in response to the same concept, like "Spider-Man," whether it appeared as a photograph, a drawing, or text. This research showed how artificial neural networks may operate similarly to the human brain. But in 2020, Olah was one of seven OpenAI employees, including CEO Dario Amodei, to leave the company over concerns about AI safety. That group went on to cofound Anthropic, which has since become a $965 billion company (per the source) and a leading voice for building AI responsibly.

Olah's Vatican remarks run directly contrary to the techno-optimist camp, most notably Marc Andreessen, who argued in his 2023 "Techno-Optimist Manifesto" that "trust and safety" and "tech ethics" were part of a demoralization campaign led by "enemies" against technology and life. Olah, by contrast, argued that even the most sincere intentions inside AI companies are inevitably shaped by incentives - the pressure to remain profitable, to lead research, and to navigate geopolitics. "No matter how sincerely any of us intend to do the right thing, and I believe many of us do, we will always be influenced by those incentives," he said. As a result, he warned, outside critics - including the Catholic Church, scholars, and governments - must supervise the industry and keep its moral obligations at the forefront.

The Pope's encyclical, "Magnifica Humanitas," serves as a moral framework for AI, calling for "a measured and vigilant approach" to its development and prioritizing humans over machines. Olah's alignment with that framework is no accident - it squares with Anthropic's own mission, which emphasizes safety and doesn't shy away from presenting research on the risks of AI. At Anthropic, Olah has continued to advance mechanistic interpretability, aiming to reverse-engineer AI models to identify which clusters of artificial neurons activate for what purposes and how they shape a model's outputs. In 2024, Time named him to its TIME100 AI list of the most influential people in the AI industry. "If we could really understand these systems, and this would require a lot of progress, we might be able to go and say when these models are actually safe," he told Time. "Or whether they just appear safe."

For executives and boards across the AI landscape, Olah's Vatican moment is more than a curiosity - it's a signal. The debate over AI governance is no longer confined to tech conferences and congressional hearings. It now has a moral authority from the Vatican, and a prominent industry insider who is willing to say that self-regulation is not enough. As Anthropic moves toward its IPO, the company's safety-first positioning could become a competitive advantage - or a target for investors who want faster growth. Either way, the message from the papal seat is clear: the world is watching, and the industry's ability to govern itself is very much in doubt.