Anthropic co-founder Chris Olah’s remarks at the Vatican conflated machine learning artifacts with human consciousness, undermining the company’s credibility on AI safety.
Context of the remarks
Speaking at the release of Pope Leo XIV’s encyclical *Magnifica Humanitas*, Olah pushed back on the Pope’s clear distinction between human and machine intelligence. The encyclical explicitly warns against equating AI outputs with human understanding, noting that these systems “merely imitate certain functions of human intelligence.” Rather than endorsing this position, Olah described AI models as “mysterious,” “grown” from human data, and possessing internal states that “functionally mirror joy, satisfaction, fear, grief, and unease.”
Technical inaccuracies
Olah’s framing conflates analogy with identity. Modern AI models are based on neural networks, but artificial neurons are mathematical constructs—not biological cells. The “mystery” of model interpretability stems from complexity and opaque training data, not from emergent sentience. Anthropic and its competitors face over 100 lawsuits precisely because their training data was scraped without consent, not inherited as Olah suggested. The claim that models are “grown” ignores the engineered nature of their architecture and the massive energy and water resources required for training.
Misleading anthropomorphism
Attributing human emotional states to model weight activations is technically unsound. There are no chemical or biological signals to measure; “joy” or “fear” in an AI context is at best a metaphor for output patterns. Olah’s own admission that he “doesn’t know what that means” underscores the problem. Alan Turing’s Imitation Game established in 1950 that a system can simulate human responses without possessing human experience. An imitation is not the real thing.
Implications for enterprise AI governance
For enterprise customers, Olah’s remarks risk legitimizing the very anthropomorphism that leads to over-reliance on AI outputs and poor risk management. Treating models as partners rather than tools invites liability, regulatory exposure, and operational failures. Businesses should demand transparency in training data provenance, reject mystical narratives, and insist on rigorous evaluation of model behavior—not speculative claims about internal experience.
The core lesson remains: AI systems are powerful statistical engines, not thinking beings. Olah would have served his company and the industry better by endorsing the Pope’s clear-eyed distinction rather than hallucinating ghosts in the machine.
