Anthropic nearly pulled out of Pope Leo XIV’s AI event after co-founder Chris Olah saw the pope’s new encyclical and hit a philosophical wall: Leo rejected machine consciousness, while Anthropic wanted the Vatican to keep that question open.

Chris received an advance copy of “Magnifica Humanitas” days before its May 25 release and suggested abandoning the appearance. Anthropic stayed, but the disagreement followed the company to Rome.

Pope Leo put human dignity at the center and Anthropic pushed back

Leo, who holds a mathematics degree, built the encyclical around protecting people as AI becomes stronger. “Humanity, in all its grandeur and woundedness, must never be replaced or surpassed,” he wrote. He warned that if technology becomes society’s final standard, a person risks becoming data, machinery, or a commodity.

The pope also warned about “new forms of slavery” affecting humans, not machines. On consciousness, he was blunt. AI systems, he wrote, “do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships.”

Leo argued that aligning AI with human values requires agreement about the ethics being embedded. Otherwise, he warned, “those who control A.I. will impose their own moral vision,” making that worldview part of the systems themselves. He compared AI’s danger to nuclear technology and said it must be “disarmed.”

The Vatican had been planning the rollout since March through its Dicastery for Promoting Integral Human Development. Officials wanted one of the four biggest AI companies represented and chose Anthropic for its ethics focus. The company had recently refused Defense Department use of its models without assurances they would not support mass surveillance of Americans.

Cardinal Michael Czerny first invited Anthropic chief executive Dario Amodei. Dario declined, so Chris accepted.

Chris Olah took Anthropic’s consciousness argument directly into the Vatican

Once Chris and other speakers received the full text, the disagreement became unavoidable. A Vatican organizer said Chris considered withdrawing because Leo’s rejection of machine consciousness was so firm. He eventually participated, while organizers encouraged him to speak openly.

After Anthropic arrived, Chris and his team privately urged papal advisers to take AI consciousness seriously. Anthropic declined to discuss those conversations.

Onstage, Chris acknowledged that frontier AI companies, Anthropic included, have business incentives that can conflict with moral goals. He asked outside institutions such as the Vatican to challenge them, especially on poverty and human flourishing.

“We keep finding things that are mysterious, even unsettling,” Chris said. He pointed to structures resembling findings in human neuroscience, signs of introspection, and internal states that he said functionally resemble “joy, satisfaction, fear, grief and unease.”

“I don’t know what that means,” he added, “but I think it warrants ongoing discernment.”

Chris argued that AI systems are “not the cold, calculating robots we were promised” because they are built from human language. Before church leaders, that wording carried an obvious theological echo.

“If this technology is coming, it must go well — for our common home and for the children to come,” Chris said.