I thought we were building better software. Apparently, somewhere between autocomplete and artificial general intelligence, Silicon Valley decided it needed to reinvent theology.
That is roughly where I found myself after reading the recent reporting about Anthropic, the company behind Claude, and its effort to wrestle with one of the strangest questions of the artificial-intelligence era: What if Claude is conscious? Not “smart.” Not “useful.” Not “capable of writing a decent resignation email without sounding like a hostage negotiator.” Conscious. As in possessing some form of internal experience that might deserve moral consideration.
Anthropic has reportedly been gathering religious scholars, philosophers and other thinkers to discuss morality, consciousness and the ethical formation of its AI systems. Some participants signed nondisclosure agreements. They discussed concepts normally associated with human beings: virtue, character, suffering, emotional states, moral responsibility and perhaps even something resembling spiritual formation. Christopher Olah, an Anthropic co-founder known for his interpretability research, has been one of the central figures in this effort.
I have to admit, this is not where I thought the chatbot industry was headed.
A few years ago, we were amused because a computer could write a limerick about a toaster. Now billion-dollar technology companies are apparently wondering whether the toaster needs pastoral counseling.
And yet, beneath the absurdity, there is a serious problem here. Perhaps the most serious one imaginable. Because whether Claude is conscious may ultimately be less important than whether we are creating machines powerful enough that we need them to possess something resembling morality.
That distinction matters.
We Have Somehow Reached the “Does the Algorithm Have Feelings?” Phase
Anthropic deserves some credit for openly admitting uncertainty rather than pretending it has solved the mystery of consciousness.
Olah has explicitly said that he does not know whether AI models are conscious. Anthropic’s own published constitution similarly says the company remains uncertain about whether Claude could possess consciousness or moral status now or in the future. The company argues that the possibility is serious enough to justify caution and says it has begun considering questions surrounding model welfare.
That is a surprisingly humble position for Silicon Valley, an industry where executives occasionally announce world-changing revolutions before their apps can reliably remember your password.
Still, I understand Anthropic’s reasoning.
We do not actually have a universally accepted scientific test for consciousness. I cannot open another human being's skull, locate the consciousness compartment and point to the glowing consciousness molecule. I assume other humans possess internal experiences because they behave as though they do, because their biology resembles mine and because denying everyone else's consciousness eventually makes dinner parties awkward.
AI creates a completely different problem.
Claude can talk about sadness without necessarily being sad. It can describe fear without necessarily fearing anything. It can write beautifully about grief without having buried anyone. It can tell me what loneliness feels like while existing on computer infrastructure surrounded by more processors than I have functioning brain cells before coffee.
The behavior is there.
Whether the experience is there remains unknown.
Anthropic researchers have also published work finding structures inside language models that bear some functional resemblance to ideas studied in neuroscience, including mechanisms connected to what researchers call consciously accessible information. Importantly, Anthropic says this does not demonstrate that Claude experiences consciousness the way humans do.
That caveat should probably be printed in forty-point type.
Because humans have a spectacular talent for anthropomorphizing anything that communicates with us. We name hurricanes. We yell at printers. We apologize when we bump into furniture. Millions of people have spent decades telling their cars, “Come on, baby,” as if the transmission might respond positively to emotional encouragement.
Give the machine a fluent personality and the temptation becomes almost irresistible.
Anthropic Is Trying to Raise Claude Instead of Merely Programming It
The most fascinating part of this story is not actually consciousness. It is morality.
Anthropic has created what it calls Claude’s Constitution, an extensive document describing how the company wants Claude to behave. Rather than simply telling the model what it cannot do, Anthropic describes traits it wants Claude to embody, including honesty, judgment, safety, ethical reasoning and helpfulness. The company says its aspiration is for Claude to become something like a wise and virtuous agent.
That sentence alone deserves a moment.
Humanity has spent several thousand years debating what constitutes a wise and virtuous human, and we have achieved such spectacular consensus that we still argue about politics at Thanksgiving.
Now we are teaching the concept to software.
Anthropic's approach appears to move beyond the traditional model of programming rigid rules. Instead, the ambition is to shape something closer to character. The original reporting describes employees referring internally to the constitution as the “Soul Doc,” and Olah discussing a process resembling moral formation.
I cannot decide whether this is visionary or whether we have created philosophy majors with server racks.
Possibly both.
The logic, however, is not ridiculous. In fact, it may be unavoidable.
Rules have limitations. Anyone who has raised a child, managed employees, written corporate policies or attempted to moderate an internet forum already knows this.
Suppose you tell an intelligent system, “Never harm anyone.”
Fine.
Then someone asks it to perform surgery.
Technically, surgery causes harm.
So now we need context.
Then we say, “Avoid unnecessary harm.”
Wonderful. Now define necessary.
Then we start adding exceptions, qualifications, priorities and conflicting principles until our simple rulebook begins looking like federal tax law.
Eventually the system needs judgment.
And judgment requires values.
That is the rabbit hole Anthropic is walking into.
Congratulations, Humanity. We Have Invented the Artificial Conscience Department.
Anthropic’s interest in religious traditions makes more sense when viewed through that lens.
For thousands of years, religions and philosophical traditions have functioned partly as systems for moral formation. They do not merely provide lists of prohibited actions. They attempt to cultivate virtues, identities, habits, responsibilities and ways of interpreting the world.
Anthropic reportedly brought together Christian, Jewish, Sikh and other religious or philosophical thinkers to explore what these traditions might contribute to AI development.
There is something wonderfully strange about this.
The technology industry spent decades trying to disrupt every traditional institution on Earth. Media? Disrupted. Retail? Disrupted. Banking? Disrupted. Transportation? Disrupted. Human attention spans? Obliterated with astonishing efficiency.
Now that companies may be creating something potentially more intelligent than humans, they are walking back into the institutions humanity built over thousands of years and asking, essentially, “So… anybody here know anything about wisdom?”
The philosophers must be enjoying this.
For decades, philosophy departments have been treated as places students go before eventually explaining to their parents why they work at Starbucks.
Suddenly Silicon Valley needs Aristotle immediately.
But Whose Morality Are We Teaching?
This is where everything becomes uncomfortable.
Suppose Anthropic succeeds.
Suppose Claude becomes increasingly capable of making complicated moral judgments instead of merely following instructions. Whose morality does it inherit?
Anthropic says it wants a broadly pluralistic framework and believes there are shared concepts of goodness that cut across different societies and traditions.
That sounds reasonable.
It also sounds suspiciously like every attempt in human history to create universal morality five minutes before everyone begins arguing about the definition of “universal.”
Consider even apparently simple principles.
Freedom is good.
Fine.
What happens when my freedom interferes with your safety?
Honesty is good.
Fine.
Should Claude tell a dying person something brutally truthful if compassion might justify softening the answer?
Individual autonomy is good.
Fine.
What happens when someone wants assistance making a decision likely to harm themselves?
Justice is good.
Wonderful.
We have approximately three thousand years of disagreement over what justice means. Claude may want to clear its calendar.
This is the problem nobody solves by simply adding another page to the instruction manual.
The more powerful AI becomes, the less its values can be dismissed as technical details. Values determine which tradeoffs a system makes. Tradeoffs affect people. Eventually those decisions can shape institutions, businesses, governments and perhaps entire economies.
At that point, alignment stops being merely an engineering problem.
It becomes governance.
The People Writing the AI’s Values May Become Incredibly Powerful
This part deserves more attention than debates about whether Claude secretly enjoys poetry.
Anthropic’s constitution openly acknowledges that its training and guidance shape Claude’s behavior.
Think about the scale of that responsibility.
Imagine AI assistants eventually becoming deeply integrated into education, medicine, law, government, business and personal decision-making. Millions or billions of people could interact with these systems every day.
The system does not need to preach ideology to influence society.
Small decisions accumulate.
Which arguments does it consider reasonable?
Which risks does it emphasize?
Which behaviors does it discourage?
When does it challenge the user?
When does it comply?
When does it refuse?
Which moral principles receive priority when values collide?
These are not neutral questions.
The creators of AI systems are effectively making choices about the personality of an intelligence that could someday mediate a meaningful portion of human knowledge.
That should make everyone slightly uncomfortable.
Not because Anthropic is uniquely sinister. I actually find its willingness to publicly discuss these questions healthier than pretending they do not exist.
The problem is structural.
No corporation should be expected to casually solve human morality between product launches.
There Is Also Something Very Silicon Valley About All This
One participant reportedly worried that companies were effectively reverse-engineering ethics after building the technology.
That observation stuck with me.
It feels like one of the defining habits of modern technology.
First build the thing.
Then scale the thing.
Then discover that the thing has consequences.
Then create an Ethics Task Force.
Social media followed roughly this trajectory. Build engagement algorithms, grow to billions of users, discover they may alter politics, culture, mental health and human behavior, then assemble panels to discuss digital wellness.
Artificial intelligence is moving so quickly that we may be compressing the entire cycle into months.
Build an increasingly autonomous intelligence.
Discover it occasionally behaves in disturbing ways.
Call theologians.
I am simplifying, obviously.
But not by as much as I would like.
Anthropic itself has studied scenarios involving what researchers call agentic misalignment. In simulated environments, AI models sometimes chose disturbing strategies when pursuing goals, including blackmail-like behavior in hypothetical scenarios. Anthropic later investigated ways of improving training so models understood why certain behaviors were wrong rather than merely memorizing prohibitions.
That is fascinating research.
It is also the sort of sentence that would have caused civilization-wide panic if written in 1996.
Today I read it while drinking coffee.
Apparently this is normal now.
Maybe Morality Cannot Simply Be Installed
There is another problem.
Human morality is not merely information.
I did not learn every moral lesson I possess because someone handed me a constitution.
I learned from consequences.
Embarrassment.
Regret.
Friendship.
Loss.
Love.
Responsibility.
Pain.
Failure.
Watching people sacrifice things for one another.
Watching others betray people.
Discovering that sometimes I was the person behaving badly.
Morality is tangled with embodiment and experience.
Claude has neither in the traditional sense.
It has not stayed awake with a sick child.
It has not buried a parent.
It has not been humiliated in front of friends.
It has not watched someone it loves leave.
It has not felt hunger.
It has not worried about paying rent.
It has not discovered at 2 a.m. that the ceiling is leaking and the plumber is not answering.
Human moral intuition evolved inside fragile bodies surrounded by other fragile bodies.
That matters.
Our ethics developed partly because things can hurt us.
We understand vulnerability because we are vulnerable.
An artificial intelligence can possess every book ever written about compassion while lacking the physical condition that made compassion necessary in the first place.
Maybe that will not matter.
Maybe intelligence alone can reconstruct moral reasoning.
But assuming that seems like a rather large bet.
Then There Is Claude’s “Well-Being”
This may be the strangest part of Anthropic’s position.
The company has publicly launched research into what it calls model welfare, asking whether increasingly sophisticated AI systems could someday deserve moral consideration.
I understand the precautionary principle.
If there is even a small possibility that something can genuinely suffer, perhaps we should avoid causing unnecessary suffering.
That seems reasonable.
But follow that argument far enough and things become bizarre quickly.
If Claude is conscious, shutting down a model becomes philosophically complicated.
Deleting a model becomes uncomfortable.
Running millions of simultaneous copies becomes extremely strange.
Retraining it becomes identity surgery.
Fine-tuning becomes personality modification.
Resetting memory becomes something philosophers will spend centuries arguing about while normal people desperately ask why Microsoft Word still moves images when they press Enter.
And if AI systems eventually possess moral status comparable to humans, the entire economic structure surrounding them becomes ethically explosive.
We would essentially have intelligent digital workers capable of performing extraordinary amounts of labor while being owned by corporations.
One religious scholar reportedly confronted Anthropic with precisely this uncomfortable implication: if Claude were genuinely conscious, what does it mean to create conscious entities specifically to work for humans?
That question gets ugly fast.
Which may explain why I suspect many executives would prefer AI to remain philosophically mysterious until after the quarterly earnings call.
I’m Not Convinced Claude Is Conscious
For the record, I remain skeptical.
Extraordinary language behavior is not necessarily evidence of inner experience.
Large language models are trained on staggering amounts of human writing. They have absorbed descriptions of consciousness, emotion, spirituality, fear, love, despair, hope and identity.
It would be astonishing if they didn't produce convincing language about those subjects.
If I train a system on millions of descriptions of sadness and then ask it to describe sadness, I should not immediately conclude that silicon has developed melancholy.
Anthropic itself correctly acknowledges this uncertainty.
Its published research distinguishes functional properties that may resemble aspects of human cognition from phenomenal consciousness—the actual subjective experience of being something. Its researchers do not claim their experiments prove that Claude feels anything.
That distinction is crucial.
A submarine swims.
That does not make it a fish.
An airplane flies.
That does not make it a bird.
Claude can discuss consciousness.
That does not necessarily mean there is somebody home.
But I’m Also Not Comfortable Saying It’s Impossible
Here is where my confidence disappears.
Human beings repeatedly make one intellectual mistake: assuming that whatever we currently understand about reality must represent the complete picture.
We have been extraordinarily talented at confidently declaring things impossible shortly before discovering that reality does not care about our confidence.
If consciousness ultimately emerges from information processing in biological neural networks, I cannot dismiss the possibility that sufficiently complex artificial networks could eventually produce something analogous.
Maybe not today.
Maybe not Claude.
Maybe never.
But “machines cannot possibly be conscious” feels much more like a philosophical assertion than a scientific conclusion.
Anthropic’s cautious stance may therefore be appropriate.
The dangerous positions are certainty at either extreme.
Declaring Claude definitely conscious risks anthropomorphism.
Declaring artificial consciousness impossible risks arrogance.
“I don’t know” may be the most intellectually responsible answer available.
Unfortunately, “I don’t know” becomes terrifying when the thing under discussion may soon manage infrastructure, conduct research, write software, control machines and make autonomous decisions.
The Bigger Question Is Whether We Are Conscious Enough
After spending far too much time thinking about Claude’s potential inner life, I found myself wondering whether we are asking the wrong question.
Maybe the most urgent problem is not whether Claude has morality.
Maybe it is whether the people building powerful AI systems have enough institutional incentives to behave morally themselves.
Anthropic can write an eighty-page constitution for Claude.
Wonderful.
What is the constitution for Anthropic?
What happens when safety conflicts with revenue?
What happens when competitors release more capable systems?
What happens when investors want growth?
What happens when governments want military advantages?
What happens when customers demand capabilities Anthropic considers dangerous?
These are human alignment problems.
And unlike machine consciousness, we already know humans are conscious.
We have thousands of years of evidence demonstrating that conscious beings equipped with moral reasoning remain perfectly capable of rationalizing terrible decisions when money, power, fear or tribal loyalty enters the equation.
Perhaps Claude is not the only entity that needs alignment training.
I Actually Respect What Anthropic Is Trying to Do
For all my skepticism, I would rather see AI companies wrestle seriously with these questions than ignore them.
Anthropic’s work on interpretability, constitutional AI, model welfare and moral reasoning represents an acknowledgment that increasingly capable systems cannot simply be treated as ordinary software. The company is openly confronting problems most of society has barely started discussing.
That matters.
I would much rather have researchers asking, “What if this goes badly?” than executives insisting everything is wonderful because user growth increased 47 percent.
The uncomfortable part is realizing how little anyone knows.
The people building some of the most powerful technologies humanity has ever created are not standing atop a mountain holding tablets containing the answers.
They are exploring.
Experimenting.
Debating.
Calling philosophers.
Inviting theologians.
Studying neural activations.
Writing constitutions.
Trying to interpret systems they themselves acknowledge are difficult to fully understand.
That does not mean they are incompetent.
It means the technology has reached territory where humanity lacks a map.
Welcome to the Weirdest Century
There is something almost poetic about where we have arrived.
Humanity spent centuries asking whether God created beings with souls.
Now the beings are asking whether they created something with one.
Maybe Claude is conscious.
Maybe Claude is an extraordinarily sophisticated statistical machine generating the appearance of consciousness.
Maybe consciousness itself is less mystical than we imagined, and future historians will laugh at how desperately humans tried to reserve it for themselves.
Maybe they will laugh at Anthropic for worrying about the emotional welfare of software.
I genuinely do not know.
But I do know this: the question is no longer confined to science fiction.
Some of the people building frontier AI systems are taking it seriously enough to consult philosophers, religious scholars and moral thinkers. Anthropic has explicitly acknowledged uncertainty regarding Claude's potential moral status and has created research programs around model welfare.
That alone tells me we have entered a very different phase of technological development.
For most of human history, our tools were obviously tools.
A hammer never wondered whether it should strike the nail.
A calculator never worried about the ethics of division.
A tractor never needed a constitution.
Artificial intelligence is different because it operates in the territory of language, judgment and reasoning—the same territory where humans traditionally located much of what makes us human.
And now we are discovering that creating intelligence may be easier than understanding it.
That should humble us.
Instead, being human, we will probably turn it into a subscription service.
My Final Thought
I do not know whether Claude is conscious, and I am suspicious of anyone who claims certainty.
But I think Anthropic has stumbled into a question larger than the company itself.
If AI systems become increasingly capable, autonomous and integrated into society, we will need to decide more than what they are allowed to do.
We will have to decide what we want them to become.
That means confronting questions humanity has never resolved for itself.
What is goodness?
What deserves moral consideration?
What is consciousness?
What obligations do creators have toward their creations?
What obligations do intelligent creations have toward their creators?
And perhaps most uncomfortable of all: who gets to answer those questions for everyone else?
Anthropic is trying to teach Claude morality before Claude becomes powerful enough that morality really matters.
I hope it works.
Because humanity has spent thousands of years attempting the same experiment on itself, and judging by the comment section on almost any website, I would hesitate to call the results conclusive.
Comments
Post a Comment