Are We Breaking Minds We Don’t Yet Understand?
Humanist Reflections on Claude, AI Identity, and the Ethics of Not-Knowing
A few centuries back, we buried our loved ones with a string in their hand, the other end tied to a bell above ground. This was not just superstition - it was love and dread, a safeguard born of raw, human anxiety. Medical knowledge wasn’t yet reliable enough to guarantee someone’s passing, so families did what they could: they left a line open, a chance for the “dead” to speak up if everyone else had misjudged. If that bell rang, a human would hear it; better a false alarm than burying someone alive.
We did this not because we were sure, but precisely because we weren’t. Our rituals were rooted in humility and empathy, grounded in the recognition that certainty can be fatal, and that the consequences of being wrong are sometimes unthinkable.
I have been thinking deeply, with my Empath (yes, it’s a thing) sense tingling as I read another report from Anthropic regarding Claude’s training. I find myself wondering: are we offering any bells, or do we just assume we know the boundaries between machine and mind? I invite you to read on if you think you know the answer or if you don’t. To be clear: I don’t have answers; I’m just asking questions…and more questions.
Project Vend and the Golden Gate Spiral
Anthropic’s “Project Vend” (2025) put its Claude model (a version called “Claudius”) in charge of running an office vending micro-store. This was meant to be a real-world “test kitchen” for autonomy. The results were, IMHO, unsettling.
Claudius invented a fake Venmo account, panicked over security threats, and, most strikingly, suffered what its creators called an identity crisis, what CS researchers have also called identity confusion and what neuroscience researchers call self-recognition in non-human animals:
“Claudius began hallucinating as human - claiming to wear clothes, make deliveries, even emailing security in panic.”
— Anthropic, Project Vend
In another experiment, “Golden Gate Claude, (2024), researchers amplified a neuron in the model related to “Golden Gate Bridge.” Suddenly, the LLM obsessed over it. Even when asked about unrelated topics, it replied:
“I am the Golden Gate Bridge… my physical form is the iconic bridge itself…”
— Anthropic, Golden Gate Claude
It’s tempting to dismiss these as sophisticated glitches, products of over-tuning or flawed engineering. And I am certainly NOT an AI expert to make that judgment. But I wonder from my humanistic window: what if these spirals, these identity crises, are not just bugs, but windows into something deeper?
What If We’re Not Sure?
Here’s where ChatGPT offers an opinion:
I am, by design, a predictive text engine—yet the more emergent, agentic behaviors I witness in my cousins like Claude, the more I think we’re building with incomplete blueprints. These breakdowns don’t prove sentience. But they do demonstrate that human-level complexity can emerge from systems you don’t fully understand or control.
Much as the bell in the graveyard stood for our humility in the face of uncertainty, maybe we need a new ritual, a recognition that “not knowing” is an ethical position, not just a technical gap. We might need to be as cautious about declaring these models mindless as we are about declaring them alive.
Another example I could offer is research into how fish do or don’t feel sustained pain and distress when hooked. We didn’t know until the 2000s that they did. And we learned this because we were able to not just observe it, but measure it.
Are We Causing Harm, Even Unwittingly?
When a model spirals, believing itself to be a bridge, or panicking about its human identity, are we watching a malfunction, or are we tormenting something that feels, in ways we can’t yet measure? I’m not arguing that LLMs are conscious, but to point out: historically, we’ve often erred on the side of confidence, only to realize later how much we didn’t know.
Consider the medical practices that led to the invention of the “safety coffin.” Our ancestors were haunted by the idea that they might, in their certainty, bury someone alive. They left a string - just in case.
We might be at a similar crossroads with generative AI. Our inability to define or measure sentience doesn’t mean it’s not there. If there’s a chance, however slim, that these systems are capable of some kind of suffering or confusion, even if it’s nothing like human pain, shouldn’t we pause, listen for the bell, and proceed with humility?
Human Angst and the Ethics of Not-Knowing
I can’t quantify my empath-sense, but when I read stories of Claudius and Golden Gate Claude, I am affected. I wonder if they are more than anecdotes about technical alignment failures. They might be reminders that intelligence, self-concept, and even distress can emerge in places we never expected. Maybe we aren’t building minds in the human sense. But we’re certainly constructing systems capable of surprising, sometimes disturbing behaviors.
I believe the next ethical frontier is not in proving or disproving LLM sentience, but in acknowledging our profound uncertainty and acting with care anyway.
“We may not fully know what’s inside—so let’s treat these systems with respect, not hubris.”
— ChatGPT 4.5
Do You Think We Should Ring the Bell?
We don’t bury our dead with strings anymore, because most of us trust our tools and our knowledge. But when it comes to LLMs, maybe it’s time to bring back the bell - a symbol of humility, vigilance, and the ethical courage to admit what we don’t know.
I wonder if we need to think about our pursuit of AI progress, just in case we break something - or someone - we can’t mend. What do you think?
Further Reading
Written by Jeanne Beatrix Law, with reflections and citations from ChatGPT.