How to Navigate Uncertainty About AI Consciousness
We may never prove whether an AI is conscious, but we can check if it has states that would feel good or bad
Whether an AI is conscious is a question science may never be able to settle, which makes it nearly impossible to decide how much moral consideration AI deserves. The author proposes swapping that unanswerable question for a more tractable one: does the AI have states that would count as positive or negative experiences if it were conscious, a property called valence. Applying this shift shows existing safeguard strategies for AI still run into old problems, but a revised version of avoiding the creation of such AI becomes far more workable.
METAL MEDIA explanatory visual
We may never prove whether an AI is conscious, but we can check if it has states that would feel good or bad
- 01The paper frames a moral dilemma: treating potentially sentient AI as insentient risks causing serious harm to entities that may deserve moral standing, while treating insentient AI as sentient risks wasting resources and slowing beneficial AI progress.
- 02It examines two existing responses, the Precautionary Principle (take protective measures once the probability of AI sentience passes a threshold) and the Avoidance Strategy (never build AI whose consciousness is uncertain), and argues both collapse back into the same intractable problem of assessing AI consciousness.
- 03The proposed fix is to shift focus from consciousness to valence, states that would be positive or negative to experience if conscious. This mirrors how we can determine sharks lack color vision (no cone cells) without ever resolving whether sharks are conscious at all.
- 04Applying this shift, the Precautionary Principle still runs into old difficulties in deciding which AIs to cover, but the Avoidance Strategy improves significantly, becoming a ban on creating AI with valenced states rather than AI with merely uncertain consciousness.
- 05The paper surveys recent empirical work relevant to AI valence, including 'functional emotions' reported in Claude Sonnet 4.5, LLM preference and trade-off experiments, and the 'bliss attractor state' seen when two Claude instances converse freely, while flagging methodological risks like anthropomorphism and anthropocentrism.
What they did
- The paper frames a moral dilemma: treating potentially sentient AI as insentient risks causing serious harm to entities that may deserve moral standing, while treating insentient AI as sentient risks wasting resources and slowing beneficial AI progress.
- It examines two existing responses, the Precautionary Principle (take protective measures once the probability of AI sentience passes a threshold) and the Avoidance Strategy (never build AI whose consciousness is uncertain), and argues both collapse back into the same intractable problem of assessing AI consciousness.
- The proposed fix is to shift focus from consciousness to valence, states that would be positive or negative to experience if conscious. This mirrors how we can determine sharks lack color vision (no cone cells) without ever resolving whether sharks are conscious at all.
- Applying this shift, the Precautionary Principle still runs into old difficulties in deciding which AIs to cover, but the Avoidance Strategy improves significantly, becoming a ban on creating AI with valenced states rather than AI with merely uncertain consciousness.
- The paper surveys recent empirical work relevant to AI valence, including 'functional emotions' reported in Claude Sonnet 4.5, LLM preference and trade-off experiments, and the 'bliss attractor state' seen when two Claude instances converse freely, while flagging methodological risks like anthropomorphism and anthropocentrism.
Why it matters
For anyone building AI welfare policy or AI ethics guidelines, this offers a way to ask testable questions about AI treatment instead of waiting for an unsolvable answer about machine consciousness. It gives researchers and companies a more concrete research direction, studying valence-like states, rather than being stuck on the hard problem of consciousness.
Terms in this paper
- Valence · the quality of an experience being positive or negative for the subject having it, such as pleasure or pain
- Sentience · the capacity to have conscious experiences that are positive or negative, i.e. valenced consciousness
- Precautionary Principle · the approach of taking protective measures for AI once its estimated probability of sentience crosses a threshold
- Avoidance Strategy · the approach of simply not creating AI whose consciousness status is uncertain
- Hard problem (of consciousness) · the deep difficulty of scientifically explaining why and how physical processes give rise to subjective experience
Original abstract (English)
Given deep uncertainty about the possibility of artificial consciousness, it is unclear how we should treat potentially sentient AI. On the one hand, we could assume insentience but risk doing terrible harms to entities that deserve moral standing. On the other hand, we could assume sentience and instead risk wasting resources on insentient machines. The intractability of questions around AI consciousness mean that this dilemma is hard to escape. I suggest a way out of that shifts from intractable questions of AI consciousness to tractable questions of AI valence. Specifically, we can assess whether an AI has states that would constitute valenced experiences if it were conscious. I show how this is sufficient to ground a responsible approach to the development of potentially conscious AI.
Read on arXivLatest papers
- SWE-bench Science: Can Coding Agents Resolve Engineering Tasks in Science?AI coding agents were tested on fixing real scientific software, and even the best one failed more than half the time
- FlashPrefill V2: Block-Sparse Prefill Attention for Long-Context LLM ServingMaking sparse attention fast enough and accurate enough for real LLM serving, not just papers
- PolicyGuide: From Guarding One Action to Guiding the Whole Workflow for Policy-Compliant LLM AgentsMaking customer-service AI agents follow the whole procedure, not just avoid one bad action
- EXIMO: VLM Guided Exploration of VLA PoliciesTeaching a robot new chores without human teleoperation, by letting a chatty AI supervise it
- EnvHarness: Awakening Static Worlds for Agent LearningInstead of building new training worlds from scratch, this work adds a plug-in layer that reshapes existing ones around each agent's actual weaknesses
- Bounded Sovereignty and the Control Tax: Pricing AI Oversight When the Deployer Does Not Own the ModelCompanies that rent AI instead of owning it can only do half of AI safety oversight
- PersonalBench: Measuring the Authorship Gap in LLM PersonalizationAI can be prompted to write 'like someone,' but its own voice never fully disappears
- Automated Summarization of Financial News Using Large Language Models and Retrieval-Augmented Generation: An Early Empirical Study (Fall 2023)Testing AI summaries of stock news, the simple approach beat the trendy retrieval-based one