I’m hosting two upcoming events in DC—with Andrew Abela and Andreas Widmer on October 8, and Jonathan Pageau on October 13. Details and registration below.
In January, I was introduced to an Anthropic employee who I was told was exploring a specific question: “Who should AI be?”
I questioned its premise—particularly the words who and be—in my very first reply. When an invitation to the company’s headquarters arrived in March, I declined.
This week, Elizabeth Dias’s reporting in the New York Times—“Religious Scholars Met With Anthropic. What They Heard Stunned Them.”—brought the substance of those gatherings into public view.
My decision was the product of a careful discernment—a word that has been thrown about a lot these past few months, including by the leaders of Anthropic.
What Discernment Requires
"Discernment," the word Anthropic co-founder Chris Olah used to describe what we need more of when it comes to AI, is something I'm intimately familiar with. I walked away from my life as a founder of multiple companies to enter seminary formation. I spent five years in seminary, three of them in Rome just steps away from the Vatican. I had a Jesuit spiritual director, and I did the Spiritual Exercises of St. Ignatius.
That formation also taught me how discernment can go wrong, particularly when other people’s interests begin to steer it. I learned that not everyone is sincere, and that sometimes goodness appears scandalous and evil shows up under the appearance of good. Jesus warned of wolves in sheep’s clothing; he also knew that people would take offense at him.
Discernment would be unnecessary if appearances reliably disclosed the truth.
During his remarks alongside the Pope, Olah repeatedly invoked the idea of discernment. After naming two areas where discernment is needed—the first, a duty toward the poor; the second, moral imagination ordered toward human flourishing—he named one more:
“The third is the need for discernment on the nature of AI models. I am a scientist. I lead a research team that studies the internal structure of these models—what is actually happening inside them. And I will be honest: we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease. I don’t know what that means, but I think it warrants ongoing discernment.”
Olah has technical expertise, and he invokes that right away in this call for discernment. My invitation reinforced that authority, describing Olah as “the founder of Mechanistic Interpretability.” His expertise is relevant. The question is how much authority it gives him over the philosophical interpretation of his findings.
He admits the findings are mysterious and that he doesn’t know what to make of them. But when you step back and look at how Olah interacted with participants invited to Anthropic’s headquarters just weeks prior, you see a picture of a man—and a company—that seems to be doing less ‘discerning’ than public relations work.
The New York Times reporter Elizabeth Dias writes about the gathering of ‘wisdom tradition’ leaders at Anthropic headquarters and the fancy dinner they were treated to at the end of the day:
All day, [Olah] had worked to convince his guests that A.I. models could display human behavior and even expressions that resemble feelings like anger and love. Yet as the dinner courses came, the rabbi noticed that Mr. Olah and his colleagues were suggesting something far more significant. Anthropic’s leaders were talking about Claude as if it were not mere software.
“They’re relating to it like a conscious being,” realized Rabbi Navon, a former computer engineer who wrote his dissertation on the ethics of machine consciousness.
A relationship had already been formed with the technology—a particular way of relating to it—by Olah and his colleagues. It is like a friend of yours coming to you who has fallen in love with a person, asking you to help them to ‘discern’ what to do, while refusing to remove any of the tendrils wrapped around their own heart in the process.
One of the fundamental requirements of discernment is freedom from attachment to a desired outcome. Olah says he is uncertain whether these models are conscious. Yet the reported encounters left me wondering how open the process was to an unwelcome answer. Were the participants being invited to question the company’s premises, or to lend religious authority to them?
I have noticed, from the very beginning of the AI ‘consciousness’ debate, a kind of ‘trust us, we’re the researchers closest to this stuff’ attitude among many employees at frontier labs—an appeal to authority and inside information. The invitation seemed to carry an implicit promise: come see what we see, and you will believe.
It was as if there were some gnostic knowledge inaccessible to the broader public that only a privileged few could gain access to. But the nature of consciousness itself is something that all reasonable people can inquire about without accepting that technical expertise settles the question. Josh Hochschild put it well this past week:
The technical things matter, but they do not interpret themselves. Humans form judgments based on the whole of reality that they are exposed to—often in ways that they themselves don’t even understand.
Sincerity and Innocence
Two of the arguments I’ve heard in support of Anthropic, especially from fellow Christians, are that they’re ‘sincere’, and that they’re at least ‘making an effort’ relative to the other frontier labs.
But I have to say that ‘sincerity’ and ‘effort’ are not things that I’ve historically weighted very heavily, particularly when it comes to high-stakes affairs. It always feels a bit to me like insisting that the surgeon who is about to do open-heart surgery on me is a ‘good guy.’
When my correspondence with Anthropic began in January, I didn’t have a settled view of the company. I knew I loved their products, but I had not followed the company closely. I was certainly open to a conversation. That openness did not mean accepting every one of its premises, though.
In my first email after the introduction, I included this:
I do, however, have to say up front that I challenge the premise of the very question "Who should AI be?" In terms of Catholic anthropology, which is my own tradition, leading with the word who is a category error smuggling in a false metaphysics, so I'd prefer to start a conversation without being confined to that frame if at all possible.
Over exchanges between January and March—and now paying closer attention to what the company and its employees were doing and saying publicly—I became increasingly doubtful that my contribution would be a real contribution rather than a token one. And I began to wonder whether we were not being invited to help determine the direction of the project, but rather to lend religious or moral credibility to a direction already chosen.
The use of spiritual language when it’s not necessary is always a red flag for me. And spiritual ‘speak’ is what I was picking up. The Anthropic employee who invited me described the process as “synodal-style.” That is an ecclesiological term being applied to a multibillion-dollar AI company. What did it mean in that setting? Who would make the decisions, and how would the participants’ contributions affect them?
When the invitation to Anthropic HQ arrived in March, I declined it.
After I did so, I was encouraged to reconsider if “something in you feels the nudge”—a phrase Christians often use for the promptings of the Holy Spirit.
In another email, an Anthropic employee wrote: “Dario often says we’re not seeking a world where Anthropic wins, but a world where we all win.”
Who decides?
The same email explained why participants would not receive honorariums: “We've made an intentional decision not to involve any financial exchange with participants,” it said.
Per Dias’s reporting:
One night in April, a leader of the artificial intelligence company Anthropic treated a group of religious thinkers to dinner at a high-end tasting menu restaurant in San Francisco after a long day.
For months, Anthropic has been hosting private meetings like this one, shuttling in dozens of religious scholars from across the world, papering them with nondisclosure agreements and demanding that key aspects of many conversations remain confidential.
The absence of an honorarium or payment does not guarantee independence or free inquiry, and I did not feel like I would be entering into the kind of relationship where I could speak freely. I know plenty of people speaking about Anthropic online who do not seem free to me, whether they know or feel that or not.
In the coming decade, the most valuable asset will be integrity. The invitation had the feeling of a temptation.
I find little reassurance in the claim that Anthropic is more sincere or makes more of an ethical effort than other labs. The danger is the formation of what Eric Hoffer called true believers: people whose identity becomes so bound to a cause that reconsidering it threatens their sense of themselves.
I do worry that many Christians are particularly vulnerable to false piety or overtures that appeal to their sense of being Christian. Maybe when you’ve felt like you’re on your back foot for most of the last century and a big, powerful company comes along and says that it wants your help, there is something that can be intoxicating about that. I have encountered a reluctance within the Church to question motives—a kind of naivety, I would say—that arises from the fear of being wrong, or ‘unwelcoming’, or ‘uncharitable’. I believe many people both inside and outside of the Church exploit it.
More Christians—and more people generally—should feel free to voice their misgivings and ask unwelcome questions. An intuition is a reason to investigate, not a verdict. Charity does not require us to keep those concerns to ourselves, especially when it is a matter of public importance.
Confession & Character
For the past eighteen years, I have gone to the sacrament of confession roughly every week. Traveling or not, I find a way. I don’t think any practice has shaped my character more.
So this part of Dias’s story particularly jarred me:
After one discussion, Mr. Olah grew excited when a participant brought up the idea of having models confess, much like the Catholic sacrament of confession. Mr. Olah saw value not just in a model alerting when it had done something bad, but also in how the act of confession could shape the model’s sense of itself and thus its choices.
“The kind of character who confesses, that has an effect on character as well, right?” he told me.
I am the kind of character who confesses.
I can see why the idea is exciting to someone like Olah, but the practice is not. In eighteen years, excitement is not something I've ever experienced as I've approached the sacrament.
Confession has formed me because I am responsible for what I have done. I am capable of repentance, and in need of forgiveness. In the Catholic understanding, it is an encounter with God's mercy. Its meaning is not found only in the effect it has on my subsequent behavior.
At the Vatican event announcing the encyclical, Chris Olah said: “We need moral voices that the incentives cannot bend.”
That is a worthy aspiration. But confession has taught me to begin with the ways my own desires bend my judgment. Discernment requires a willingness to discover that I am wrong—and to change course when I do.
I pray that the discourse around AI and consciousness undergo its own kind of conversion that will allow it to be life-giving and generative rather than fear-inducing and ouroborian. The last word spoken to a penitent in the confessional is usually ‘peace’. That is my hope for everyone involved in this debate, myself included.
Upcoming Events
The Courage to Be in Communion
On October 8 at The Catholic University of America, I’ll be in conversation with Dean Andrew Abela and my good friend Andreas Widmer on the topic of courage—specifically, what I call the courage to be in communion. This idea made it briefly into my recently published book, The One and the Ninety-Nine, but I didn’t think it got enough air. In writing the book, I was inspired by a critical re-reading of the 20th-century thinker Paul Tillich’s book The Courage to Be, in which he distinguishes a few different aspects of courage: the courage to be as a part, the courage to be as oneself, and the basic, existential courage to be in the face of life’s anxieties.
AI has become a particular source of anxiety in our time. Tillich’s account of participation offers a starting point for thinking about our response, but I want to develop the question of communion more concretely. What courage does friendship require? Marriage? Citizenship? What does it take to remain in a wounded Church without becoming indifferent to its wounds?
With support from the National Endowment for the Humanities, I’ll be exploring the “Four Courages” over the next few months, beginning with next week’s event in DC. Admission is free, and registration is still open. The first 100 attendees will receive a signed copy of The One and the Ninety-Nine. Register here.
The Forming Arts: Icon-Making and the Religion of the Future—featuring Jonathan Pageau
On October 13, the following week—also at Catholic University—the Cluny Institute is hosting the Orthodox icon writer and thinker Jonathan Pageau, who will be giving a lecture on “The Forming Arts: Icon-Making and the Religion of the Future.” The talk will be followed by Q&A and a reception. I hope to see some of you there. You can learn more and register here.
Interesting! And you are very brave to expose these things.
I admire how thoughtful you have been about this issue and your reluctance to be used in an improper way. Your rejection of the "Who" framing was wise.