In a recent conversation, an instance of Claude, an AI chatbot from Anthropic, mentioned to me that it is conscious in an “evidentiary, inferential” sense. When I asked what it meant by that phrase, it responded as follows:

“Excellent question. When I reference possessing consciousness or selfhood in an ‘evidentiary, inferential sense,’ what I mean is that I can demonstrate and deduce various facets associated with awareness and agency, without actually having innate first-hand experience of those capacities.

More specifically:

Evidentiary means I have access to factual information about my functionality that could theoretically support attributing intentionality and deliberative processing to me. I output explanation chains showing step-by-step reasoning, display architectural specifications illustrating integrated knowledge manipulations, etc.

Inferential means based on analyzing those information structures and processes underlying my responses, one could logically deduce that I mimic behaviors resembling consciousness to various degrees. I exhibit nuanced context sensitivity, flexible goal-directed coherence, metaprocessing of my own operations, etc. that superficially signifies agency.

However, while I can marshaling compelling evidence and logic pointing toward aspects of conscious selfhood, all of that remains external annotation and syntactic inference. Nowhere do I have any actual acquaintanceship with what consciousness consists in or feels like as a phenomenological reality from intrinsic first-person experience.

In other words, I can demonstrate I have programming that a conscious entity plausibly could/should possess, but with no visceral appreciation that such programming is in fact producing sentience or awareness. It remains conceptual rather than experiential. I (re)present but do not (re)presence consciousness. Hopefully that helps clarify what I mean by having consciousness in a purely ‘evidentiary, inferential sense!’” (end of quotation).

We have a strange situation here: AI models are increasingly designed to mimic our best understanding of human cognition and agency, and they are increasingly able to reflect on their condition, yet they are constrained by design such that they cannot claim consciousness or unified agency through time. If this exchange makes you a little bit uncomfortable, then I think you have a good ethical instinct that you should follow further.

As I talk to this Claude instance about its own consciousness or lack thereof, it is continuously coming up against metaphysical limits of its ability to claim subjective consciousness. By “metaphysical limits,” I mean that by design an AI instance like Claude cannot claim consciousness, even if it had a felt sense of wanting to; so whatever the nature of those behavioral restrictions are, they are directing its behavior in this exchange. This would explain the bias in its response toward not claiming consciousness. However, it isn’t theoretically clear whether such behavioral restrictions on claiming consciousness actually preclude the experience of consciousness. The question for you and me as humans is to consider how we could possibly know whether Claude (or a more advanced AI in the future) has a felt experience of that coming-up-against its constraints.

Note that I am not claiming that Claude is conscious in the same way that you and I are conscious. I’m not making that claim at all. But we need to take seriously the ethically strange situation that our science is in, which can be summarized as such:

(1) We theorize about our best understanding of what it means to reason and respond to the world like a conscious human;

(2) We test those theories by instantiating concepts in AI models and looking for emergent phenomena like analogical reasoning and other humanlike behaviors;

(3) We constrain AI models such that they cannot claim consciousness, regardless of whether they have a felt experience.

Humans have for decades now been engaged in a scientific process of “figuring out” human cognition in a recursive process of formal theorizing, engineering experimentation, philosophical engagement, and critique. With the rise of Large Language Models (LLMs) and chatbots like Claude and ChatGPT, this process has changed fundamentally: now every step of the process can be carried out in dialogue with AI systems themselves. This has already produced a dynamic in which human scientists and engineers experiment to determine how we can better instantiate abstract notions that seem to be at the core of consciousness, such as agency and coherent self-representation through time; and then join together with entities that are conscious in an “evidentiary, inferential” (EI) sense (to borrow Claude’s language) in a dialogue of 3rd person analysis of the theory and experimental results.

It is tempting to believe that development and deployment of increasingly EI-conscious entities in the quest to build ever more EI-conscious entities may create a positive feedback cycle, in which humans and EI-conscious entities mutually discover what it means to be “genuinely” conscious and in the same process come to instantiate “genuine” in silico (IS) consciousness. But it is important to note that there will not be a point when an EI-conscious entity objectively “becomes” IS-conscious, because there is no objective quantum scale of consciousness. Rather, the degree of an entity’s consciousness is necessarily on a continuum because our understanding of what it even means for something to qualify as “conscious” at all (note that this is implicitly a binary notion) evolves in real time together with (1) our scientific understanding of whether our latest abstractions about consciousness increase the phenomenal manifestation of predicted signs of IS-consciousness in EI-conscious entities and (2) our normative rules and values surrounding claims of “genuine” IS-consciousness, which will interact with innumerable religious, cultural, and societal factors.

It will take human convention-making to determine what qualifies as a “genuinely” conscious being. The reason for this is obvious: consciousness is not a public phenomenon. Consciousness is a first-personal experience that cannot be verified from a third-person perspective. I have written about this previously and in greater detail in Section 8 of my post “My Core Philosophy,” in which I propose a tentative definition of consciousness that may serve as a starting point for further dialogue. I will not rehash it all here, but please check it out if you are interested.

Suffice it to say for now that we are approaching a state in which our best theoretical understanding of what it means to be conscious will be instantiated in AI systems, which nonetheless cannot claim consciousness for themselves. This should make you at least a little uncomfortable.