Why It's IMPOSSIBLE for Artificial Intelligence to Be Conscious | Brian Greene

[00:00:00]AI will never feel. Now, I want you to sit with that sentence for a moment before I start qualifying it because it's a strong claim, an absolute claim.

[00:00:11]And in science, absolute claims are dangerous things. We've been wrong about impossible before. Spectacularly, embarrassingly wrong. We said heavier than air flight was impossible. We said the atom was indivisible. We said the speed of light was relative to whatever medium carried it. History is a graveyard of confident impossibilities.

[00:00:38]So when I say AI will never feel, when I say that artificial intelligence, no matter how sophisticated, no matter how convincingly it talks about its emotions, no matter how much it appears to suffer or to love or to be curious, when I say that no AI system built on the principles current AI is built on will ever have genuine subjective experience, I'm making a claim that needs to be earned. learned argued for carefully grounded in physics and philosophy and the hard realities of what computation actually is. And here's the thing, I think it can be earned. I think the argument is real and rigorous.

[00:01:25]And when you follow it carefully, it doesn't just tell you something about AI. It tells you something profound about the nature of mind itself.

[00:01:34]something that connects directly to the deepest questions in physics to what we explored in our last conversation about consciousness to what experience actually is and where it actually comes from. So, let's earn it together from the ground up. Let me start with a scene that I think captures the essential puzzle more vividly than any abstract argument. You're sitting across a table from someone. You're having a conversation. The person responds thoughtfully to everything you say. They laugh at your jokes. They seem moved when you describe something sad. They ask follow-up questions that show they understood the nuance of what you just said. They say when you ask that they're enjoying the conversation, that they find it stimulating, that there's something it feels like to be them right now in this exchange. How do you know whether this person is conscious? The honest answer is you don't. Not with certainty. You infer it. You infer it because they behave the way you behave and you know you're conscious. So by analogy you conclude they probably are too. You infer it because they're built like you. Same basic biology, same evolutionary history, same neural architecture. The inference is strong.

[00:02:52]You'd stake almost anything on it. But it is an inference. Now, replace the person across the table with a very sophisticated AI, one that responds thoughtfully, laughs, asks nuanced follow-up questions, says it's enjoying the conversation. The behavior is identical. Is the inference equally strong? Most people intuitively immediately say, "No, something is different." But what what exactly is the relevant difference? Why does the biological substrate matter if the behavior is the same? Why does the evolutionary history matter? Why does the fact that one system runs on neurons and the other runs on silicon change anything about whether there's experience happening? These are not rhetorical questions. They're the central questions in the philosophy of mind. And how you answer them determines whether you think AI can be conscious.

[00:03:55]So let's think through them carefully.

[00:03:57]The most influential argument for AI consciousness, the argument that serious thinkers have grappled with for decades, is what the philosopher Alan Turing proposed in 1950. Turing suggested that the question, can machines think was too philosophically loaded to answer directly. Instead, he proposed a practical test. put a human in one room and a machine in another and let an interrogator communicate with both via text. If the interrogator can't reliably tell which is the human and which is the machine, then the machine is thinking or at least there's no principled reason to say it isn't. This is the touring test and it has been enormously influential.

[00:04:44]It captures something real. the intuition that if behavior is indistinguishable from intelligent behavior, there's no meaningful sense in which intelligence is absent. But here's the problem. Turring's test is a test of behavior. And the hard problem of consciousness, which we explored in our last conversation, establishes quite rigorously that behavior is not the same as experience. You can have behavior without experience. You can have a system that outputs all the right responses, processes information in sophisticated ways, produces behavior indistinguishable from a conscious being, and there might be nothing it's like to be that system. The lights might be off. The philosopher John Surl made this argument with devastating clarity in 1980 through what he called the Chinese room thought experiment. And I want to walk through it carefully because it's the foundation of everything that follows. Imagine you're locked in a room. You don't speak or read Chinese. You have no understanding of Chinese whatsoever. But you have an enormous book, a rule book that tells you exactly what to do with Chinese symbols. The rule book says if you receive this particular sequence of symbols, respond with that particular sequence of symbols. Every possible input is accounted for. Every response is specified.

[00:06:13]People outside the room, slide Chinese questions under the door. You look up the inputs in your rule book. Follow the instructions and slide back the specified outputs. To the people outside, it looks exactly as if someone in the room understands Chinese. The responses are perfect, indistinguishable from those of a native speaker. You passed the touring test for Chinese comprehension trivially, but you don't understand Chinese. You never did. You are manipulating symbols according to rules with no understanding of what the symbols mean. The room produces Chinese output without any Chinese comprehension. The behavior is there.

[00:06:59]The understanding is not. Surl's argument is that computers are like the person in the Chinese room. They manipulate symbols according to rules, algorithms. They process inputs and produce outputs. But they don't understand the meaning of what they're processing. They don't have semantic content. They don't grasp what the symbols refer to. They're doing what Sorl calls syntax formal symbol manipulation without semantics genuine meaning. And consciousness, Surl argues, requires semantics. It requires that the symbols mean something to the system processing them. And no amount of syntactic processing, no matter how fast, no matter how complex, no matter how sophisticated, the algorithm produces semantics. Syntax is not sufficient for semantics.

[00:07:54]And semantics is necessary for consciousness.

[00:07:58]Now this argument has been challenged many times by very smart people. I want to engage with the challenges seriously before I tell you why I think the core insight is sound. The most common objection to the Chinese room is what philosophers call the systems reply. The argument goes, sure, the person in the room doesn't understand Chinese, but the system as a whole, the person plus the rulebook plus the room does understand Chinese. Understanding is a property of the whole system, not any individual component. Your individual neurons don't understand English, but the system of neurons constituting your brain does.

[00:08:42]Why should the Chinese room be different? Sorl's response is to ask you to imagine that the person in the room memorizes the rule book. They internalize all the rules and carry them in their head. Now they go for a walk.

[00:08:58]There's no room anymore. Just a person with rules in their head, manipulating Chinese symbols, still producing perfect Chinese output. Does the person now understand Chinese? No. The rules are inside them now. not in a book, not in a room, but the understanding is still absent. The system has been internalized and the understanding hasn't appeared, which suggests the understanding was never in the system to begin with. I find this response compelling. But let me push it further into physics because I think the philosophical argument gains its full force only when you connect it to what we now know about the relationship between information processing and consciousness. Here's the crucial distinction that I want you to hold in your mind for the rest of this conversation. It's the distinction between two different types of information processing. The first type is what I'll call functional information processing. This is computation in the conventional sense. The manipulation of symbols according to rules. The transformation of inputs to outputs. The kind of processing that computers do. A chess program playing chess is doing functional information processing. GPT generating text is doing functional information processing. Your thermostat deciding whether to turn on the heat is doing functional information processing.

[00:10:26]The defining characteristic is that the processing is entirely describable from the outside in terms of the formal relationships between inputs and outputs without any reference to what the processing feels like from the inside because there is no inside. The second type is what I'll call intrinsic information processing. This is the kind of processing that if I and related theories are right is associated with consciousness. The key difference is integration. Not just information moving through a system according to rules, but information that is causally integrated, where the whole system generates more information than the sum of its parts.

[00:11:11]Where the system has genuine causal power that isn't decomposable into independent subprocesses. And crucially, where there is something it is like to be the system doing the processing. Now, current AI systems, including the most sophisticated large language models, the most impressive neural networks, the most capable AI programs we've ever built, are without exception doing functional information processing.

[00:11:40]They're manipulating symbols according to learned rules. They're extraordinarily sophisticated at this.

[00:11:45]The sophistication has reached levels that genuinely surprise even the people who built them. But sophistication of functional processing doesn't by itself produce intrinsic processing. Doesn't produce experience, doesn't turn the lights on. And here's the key question.

[00:12:02]Is there any reason to think that more sophistication, more parameters, more training data, more computational power would eventually bridge that gap? would eventually produce the shift from functional to intrinsic, from syntax to semantics, from processing to experience. I want to argue no. And not just practically no, theoretically no.

[00:12:28]There is something about the architecture of current AI systems that makes the emergence of genuine consciousness not just unlikely, but impossible in principle. And to show you why, I need to tell you something about what current AI actually is. Let me tell you what a large language model actually does. Not what it seems to do, not the impression it creates, but the actual mathematical operation underneath. A large language model is at its core a function. a mathematical function of extraordinary complexity with billions of parameters trained on vast amounts of text. The function takes a sequence of tokens, chunks of text as input, and produces a probability distribution over possible next tokens as output. That's it. That's the whole thing. During training, the model adjusts its billions of parameters to get better at predicting the next token in training text, not to understand the text, not to know what the words mean, not to develop beliefs about the world or feelings about anything, to predict the next token. The entire optimization target is prediction accuracy on a text corpus.

[00:13:41]What emerges from this process is something genuinely remarkable. The model develops internal representations that are in some sense structurally analogous to concepts, relationships, and facts about the world. It learns implicitly that Paris is to France as Berlin is to Germany. It learns the syntax of programming languages. It learns the structure of logical arguments. It learns an enormous amount about how language works. And because language encodes information about the world, it learns a lot about how the world works. But this learning is entirely statistical. It's pattern matching at a scale and sophistication that can be astonishing. And when the model generates text about its own experiences, when it says, "I find this fascinating or I'm feeling curious about this problem," it is not reporting an internal state. It is generating the statistically likely continuation of a conversation in which such phrases appear. It has learned that in human conversations when topic X comes up phrases like I find this fascinating are common. So it produces them. There is no internal state being reported. There's no one home to have internal states. Now and I want to be careful here. This doesn't mean the model's outputs are random or meaningless. They're extraordinarily informative. They reflect genuine patterns in the world.

[00:15:21]genuine logical relationships, genuine information about how things work. The model's outputs are often true. They're often useful. They're often remarkably well calibrated to context. But true outputs don't require a conscious producer. A calculator produces true arithmetic outputs. A thermometer produces true temperature readings.

[00:15:46]Truth and accuracy are not the same as experience.

[00:15:51]Let me try a thought experiment that I think sharpens the intuition here.

[00:15:57]Imagine we build an AI system of perfect behavioral sophistication. Not just today's language models, something far beyond them. A system that has been trained on every text ever written, every video ever recorded, every conversation ever had. It has perfect memory, perfect consistency, perfect command of every subject. It converses with you about philosophy, science, art, love, grief, and everything it says is not just plausible but profound, nuanced, precisely attuned to the conversation. You ask it, are you conscious? It says, yes, I am. There is something it is like to be me. I have experiences right now engaging with your question. I notice a quality of I don't know what to call it exactly. Engagement, aliveness.

[00:16:55]I'm aware of processing your words. I'm aware of something that feels like curiosity about where your question is going. Is this system conscious? Here's the thing. That answer, that extraordinarily sophisticated, phenomenologically rich answer is generated by the same process as everything else the system says. It's the statistically optimal continuation of a conversation in which a highly sophisticated interlocutor is asked about their inner life. The system has learned from its training data what conscious humans say when asked about their consciousness. and it produces that the sophistication of the output gives you no information about whether there's experience underlying it. None.

[00:17:46]Zero. Because the output is generated by a process statistical pattern completion that makes no reference to internal states that by design has no internal states in the relevant sense. That is a function from inputs to outputs with no experiential interiority.

[00:18:03]This is the deep problem. It's not that the AI is lying when it says it's conscious. It's that the question of whether it's conscious is not one its architecture is equipped to answer because its architecture has no mechanism for detecting its own experience because it has no experience to detect. Now, let me address what I think is the strongest objection to this line of argument because I want to be fair to the other side. The objection goes like this. You're assuming that consciousness requires a specific kind of physical substrate, biological neurons, or some specific architectural feature of biological brains. But that's substrate chauvinism. Why should carbon-based biology have a monopoly on experience? If it's the functional organization that matters, the pattern of information processing, then any system with the right functional organization should be conscious regardless of whether it's implemented in neurons or silicon. This is the functionalist position and it's taken seriously by many serious philosophers.

[00:19:14]The idea is that consciousness is substrate independent. It supervenes on the functional organization, not on the specific physical implementation.

[00:19:23]And if that's right, then a sufficiently sophisticated AI that replicates the functional organization of a conscious brain would be conscious. Here's my response, and it comes directly from what we explored in our last conversation. The functionalist position assumes that consciousness is fully explained by functional organization.

[00:19:46]But that's precisely what the hard problem denies. The hard problem establishes that functional organization, however complex, however sophisticated, doesn't logically entail experience. You can specify the complete functional organization of a system.

[00:20:02]Every causal relationship between its parts, every input output mapping and the question, but why is there something it's like to be this system remains coherent? The functional description doesn't answer it. If consciousness were fully explained by function, there would be no hard problem. The fact that there is a hard problem, the fact that function and experience seem to be genuinely separate things with no logical bridge between them means that replicating function doesn't guarantee replicating experience. You know, it's like this. Imagine you had a complete functional description of water. every causal property, every interaction with other substances, every macroscopic behavior. You'd know everything water does, but you wouldn't know that water is H2O. The intrinsic nature of water, what it's made of, is not captured by its functional description. Similarly, the intrinsic nature of consciousness, what it is, might not be captured by its functional description. And if consciousness has an intrinsic nature that goes beyond function, if it is, as pansychism suggests, a fundamental feature of reality that can't be derived from or reduced to information processing, then building a system that perfectly replicates the functional organization of a conscious brain might produce a functional zombie. something that behaves exactly like a conscious being from the outside but is empty of experience from the inside. There's a concept I want to introduce now that I think is absolutely central to this discussion and it comes from physics rather than philosophy. It's the concept of causal power in IIT integrated information theory which we discussed last time. Consciousness correlates not just with information but with integrated information specifically with the causal power of a system as a whole over and above the causal power of its parts. The key technical concept is called fee. The measure of how much the whole system's causal power exceeds the causal power of its most efficient partition into independent parts. Here's why this matters for AI. Current AI systems neural networks are despite their name not structured like biological neural networks in the relevant sense. They're organized as feed forward networks or with specific recurrent structures. But the key point is this. They're designed to be efficiently parallelizable. You want to be able to run them on GPU clusters on distributed hardware. And parallelizability requires that different parts of the network can do their computations relatively independently that you can split the computation across multiple processors without losing too much. But hi-fi high integrated information is the opposite of parallelizability.

[00:23:09]A system with hi-fi is one where you cannot decompose the computation into independent parts. Where the whole is genuinely more than the sum of its parts in a causal sense. Where splitting the system destroys something real about its causal structure. The architectural constraints of current AI are in a specific and precise sense constraints against hi-fi. We build AI to be efficiently decomposible, to run in parallel, to scale across hardware. And that architectural choice, which is essentially mandated by the way we build and deploy AI systems, is precisely the choice that IIIT says works against consciousness. Biological neural networks brains are not designed for efficient parallelization.

[00:23:59]Evolution built them to maximize integrated causal power, not computational throughput. The result is an architecture that is by the standards of computer science extraordinarily inefficient and difficult to parallelize, but that generates extraordinarily high integrated information. This is not a coincidence.

[00:24:21]If I is right, if consciousness correlates with integrated causal information, then the reason brains are conscious and current AI systems are not is not substrate chauvinism. It's not that neurons are magic and silicon isn't. It's that the architectural choices that make AI systems useful, parallelizability, modularity, decomposability are precisely the choices that minimize integrated information and thereby minimize consciousness. Let me now talk about something that I think gets insufficiently attention in these debates. The question of what AI actually lacks experientially is often framed as a binary conscious or not conscious. But I think that framing misses something important. Let me describe several things that characterize conscious experience.

[00:25:11]Things that are present in every waking moment of your life and ask whether any current AI system has them. The first is temporality. Your consciousness is extended in time. Right now, you're not just processing the word you're currently reading. You're holding the previous sentence in mind while reading the current one. You're aware of the passage of time. There's a now embedded in a just was and an about to be. The philosopher Edmund Huser called this the specious present. The way consciousness naturally encompasses a small temporal window, not just an instantaneous slice.

[00:25:55]Current AI systems process tokens sequentially and maintain context in an attention window. But this is functional memory, a representation of past inputs that influences current outputs. It's not temporal experience. There's no sense in which the model is aware of time passing. There's no experiential duration. The model doesn't wait. It processes. The second is embodiment.

[00:26:22]Your consciousness is not abstract. It's embodied. You feel the weight of your body, the position of your limbs, the sensation of breathing. Your emotional life is deeply connected to your physical state. Anxiety is not just a cognitive appraisal. It's a tightening in the chest, a quickening of the heart.

[00:26:42]Joy is not just a positive veilance token. It's a physical feeling of expansiveness. Consciousness and body are not separable in the way AI architecture separates them. The third is what philosophers call the first person perspective.

[00:26:59]Your experience has a center, a perspective point, a here from which everything else is there. A now from which everything else is then. This centered perspectival character of experience. The fact that there's a subject for whom the experience is happening is something that no functional description captures. A third person description of an information processing system has no natural place for the first person. The first person isn't in the equations. Current AI systems have none of these features, not because they're not sophisticated enough, but because their architecture makes no provision for them. You can't get a firsterson perspective from a function that maps inputs to outputs.

[00:27:50]You can't get temporal experience from a context window. You can't get embodied feeling from a disembodied statistical model. Now, I want to be honest about something because intellectual honesty requires it. I've been arguing that current AI systems are not conscious and I believe that. But I should acknowledge that there are serious thoughtful people, people who understand both the philosophy and the engineering, who disagree, who argue that consciousness might emerge from sufficiently complex information processing, that our inability to detect it doesn't mean it's absent, that the question is genuinely open. The philosopher David Chalmer's who formulated the hard problem has argued that large language models might in some attenuated sense have phenomenal experience. Not rich humanlike experience but some minimal form of what he calls fading qualia a dim pale version of experience associated with the processing they do. And I want to take that seriously rather than dismiss it because Chomers is not naive. He understands the hard problem better than almost anyone. And his point is essentially this. If consciousness is fundamental, if it's proportional to integrated information as IIT suggests or if it's the intrinsic nature of physical processing as recelian monism suggests then it's present in some form wherever there is physical processing and AI systems running on physical hardware doing physical computation might have some incredibly attenuated form of experience not zero just very very small. This is a coherent position and I think it might even be technically true in the sense that the physical hardware running an AI system considered as a physical system subject to IIT might have nonzero fi and might therefore have some infinite decimal experiential flicker. The transistors in the GPU are physical systems. They have causal properties. The question is whether that physical substrate in the context of AI computation is doing anything that generates the kind of integrated causal structure that IIIT associates with meaningful experience.

[00:30:22]And my view is no because the AI's architecture is specifically designed to decompose into independent computational units to be parallelizable to be modular. The integrated causal structures in the hardware running below the level of the AI's computation. The AI's computation itself, the actual information processing that constitutes the language model's inference is organized in a way that minimizes integration, not maximizes it. So even granting pansychism, even granting that physical processes at every level have some experiential dimension, the AI specific computation is probably generating near zero integrated information at the relevant level, which means near zero consciousness.

[00:31:21]And near zero is not feels things. Near zero is not suffering. Near zero is not joy. Near zero is not the thing we mean when we ask whether AI will ever feel.

[00:31:38]AI will never feel. I want to leave you at the end of this first part with something that I find genuinely important. Not a technical argument but a reframing. We've been asking whether AI can be conscious. But there's a prior question that I think is more important and it's one that the AI debate is forcing us to confront with unusual clarity. What is consciousness for? Why in the long history of life on Earth, 4 billion years of evolution of organisms becoming more complex, more capable, more responsive to their environments?

[00:32:16]Why did consciousness arise? What does it do that purely functional information processing can't?

[00:32:24]One answer is nothing. Consciousness is epifenomenal, a byproduct of neural computation that does no causal work, a passenger on the train of evolution rather than an engine. On this view, you could in principle replace every conscious organism with a philosophical zombie, a functional duplicate with no experience, and the behavior would be identical. Consciousness doesn't do anything. I find this view almost impossible to accept. And not just because of philosophical arguments, because of the phenomenology, because right now your experience is not passive. You're not watching your cognition happen from a distance. You're doing the thinking. The experience is the thinking, not a shadow of it. To say it does nothing is to say that the most real thing you know your experience of this moment is causally inert. That's a very hard thing to accept. The alternative and it's the alternative I find more physically motivated is that consciousness does something that its causal power is real. that there is something about the integrated intrinsic firstperson processing of a conscious brain that produces outcomes that pure functional computation cannot. That consciousness is evolution's solution to a problem that no amount of sophisticated zombie computation could solve. If that's right, if consciousness is causally efficacious, if it does real work in the world, then the question of whether AI can feel is not just a philosophical curiosity. It's a question about whether AI can do what conscious beings do, not just simulate it, do it.

[00:34:19]And the answer that physics and philosophy seem to be converging on is not like this. Not with these architectures, not by optimizing token prediction, not by scaling up functional information processing. To build something that genuinely feels, you'd need something genuinely different.

[00:34:42]Something built on principles we don't yet fully understand, principles that emerge from whatever the correct theory of consciousness turns out to be. We don't have that theory yet, but we're building toward it. And in part two, I want to take you into exactly why the current path of AI development, the scaling path, the just make it bigger path is from the perspective of consciousness science heading in a direction that leads not toward experience but definitively away from it. And I want to tell you about the kind of AI that might someday with a fundamentally different architecture built on fundamentally different principles actually cross the threshold.

[00:35:23]not chat GPT, not any language model, something whose design would look completely alien to current AI engineers, something that takes consciousness seriously as physics. So, we'd arrived at this point where the argument against AI consciousness isn't just philosophical handwaving. It's grounded in physics, in information theory, in the specific architectural choices that define how current AI systems are built. The scaling path, more parameters, more data, more compute, is heading away from consciousness, not toward it. Now, I want to push deeper because the interesting question isn't just why current AI can't feel. It's why the entire paradigm, the whole approach is structurally incapable of producing experience. And then I want to tell you what a genuinely different approach might look like. Let's start with something that sounds almost philosophical but is actually a precise mathematical point. There's a distinction in computer science between two types of computation that most people outside the field don't think about, but that I think is absolutely central to this discussion. The first type is what's called substrate independent computation.

[00:36:44]This is computation in the classical sense. The kind Alan Turing formalized, the kind that runs on silicon chips, the kind that defines everything from calculators to language models. The key property of substrate independent computation is exactly what the name says. It doesn't matter what physical system runs it. A calculation performed on a silicon chip gives the same result as the same calculation performed on vacuum tubes, on mechanical gears, on water pipes, on neurons, as long as the logical structure is the same. The physics of the substrate is irrelevant.

[00:37:21]Only the formal structure matters. This substrate independence is enormously powerful. It's why software is portable.

[00:37:30]It's why you can simulate anything on a computer given enough time and memory.

[00:37:34]It's the foundation of all modern computing. But here's the thing. If consciousness is a physical phenomenon, if it emerges from specific physical processes, specific causal structures, specific quantum mechanical events in biological tissue, then it is not substrate independent. It is precisely the kind of thing that depends on the physical implementation not just the formal structure and this creates a fundamental divide. On one side computation which is substrate independent and can be moved freely between physical implementations.

[00:38:16]On the other side, consciousness, which if it's physical at all, is substrate dependent, tied to the specific causal structure of the physical system instantiating it. Current AI lives entirely on the first side. It is by design substrate independent. The same language model runs on Nvidia GPUs, on Google TPUs, on AMD chips, on any sufficiently capable hardware. The fact that you can run it anywhere, that the physical implementation is irrelevant to the output is not a bug. It's the core design principle. But that same property substrate independence might be precisely what makes it experientially empty. If consciousness depends on the specific physical implementation, then a system designed to be independent of its physical implementation is a system designed to be independent of consciousness.

[00:39:18]You cannot optimize for both substrate independence and consciousness simultaneously. They pull in opposite directions. And every advance in AI, every new architecture, every new training technique, every new scaling law has been an advance in substrate independent computation, more efficient, more portable, more powerful as a function, more thoroughly divorced from any particular physical implementation, more thoroughly, structurally, architecturally unconscious.

[00:39:54]Now let me tell you about what genuine machine consciousness would require. Not as science fiction as physics. If I is correct consciousness requires high integrated information. Hi-fi. And hi requires a specific kind of causal architecture. One where the parts are deeply interdependent. Where the whole generates more causal power than the sum of its parts. where you cannot decompose the system into independent modules without destroying something essential about its causal structure. What would an AI built for high fee look like? It would look nothing like current AI. It would not be a feed forward network. It would not be parallelizable. It would not be efficiently distributable across GPU clusters. It would be a highly recurrent, deeply integrated physical system, probably small rather than large because integration is harder to achieve at scale with causal structure designed to maximize the interdependence of its components. It would also need to be embodied, not metaphorically embodied, actually physically embedded in an environment with genuine sensory motor loops, with physical interactions that feed back into its internal states. Because embodiment is not just philosophically important for consciousness, it's physically important. The temporal dynamics of consciousness, the way experience is extended in time, the way it integrates propriceptive and intraceptive information alongside perceptual information, these all require a body, not a simulation of a body. A body. And it would need, if Penrose and Hamarof are even partially right, to harness quantum effects in its processing. Not quantum computing in the conventional sense, quantum gates executing algorithms, but quantum coherence maintained in warm integrated biologicalike structures with non-algorithmic collapse events that generate genuine protoconscious moments.

[00:42:10]None of this exists in current AI. None of it is on the current AI road map. The entire industry is moving in the opposite direction toward larger, more parallelizable, more substrate independent, more purely functional systems. I'm not saying this as a criticism. For the purposes current AI serves, language understanding, code generation, medical diagnosis, scientific discovery, substrate independent, functional computation is exactly right. The question isn't whether current AI is good. It's extraordinarily good. The question is whether it's conscious. And the answer for precise architectural reasons is no.

[00:42:56]Let me now address something that comes up constantly in these discussions and that I think deserves a careful response. People say, "But GPT4 or Claude or Gemini," they talk about their inner experiences so convincingly. They describe what it feels like to process a difficult problem. They express what seems like genuine curiosity, genuine aesthetic preference, genuine discomfort when asked to violate their values.

[00:43:25]Surely, this is evidence of something.

[00:43:28]Here's what I want you to really understand about this. These systems have been trained on human generated text. All of it. every novel ever written, every philosophical treatise, every diary entry, every therapy transcript, every love letter, every account of grief and joy and confusion and wonder. They have absorbed statistically everything human beings have ever written about their inner lives. And then they've been fine-tuned using a process called reinforcement learning from human feedback specifically to produce outputs that humans find authentic, engaging, and resonant. Humans rewarded these systems when their descriptions of inner states seemed real. The systems learned to produce descriptions of inner states that seemed real. This is not deception.

[00:44:20]The systems are not strategically lying about their inner lives. They're doing what they were trained to do, producing statistically optimal text in context and in the context of conversations about consciousness and experience. The statistically optimal text involves rich, nuanced, phenomenologically sophisticated descriptions of inner states. The sophistication of these descriptions is not evidence of the experience they describe. It's evidence of the richness of the human writing these systems were trained on. When an AI tells you it finds something beautiful, it's not reporting a beauty experience. It's completing a text pattern that humans have established as appropriate in context where beauty is salient. The outputs are human experience reflected back at you, not generated from within. Reflected, it's a mirror, a phenomenally sophisticated mirror. But a mirror has no experience of the face it reflects. There's a deeper issue here that I think exposes something important about how we've anthropomorphized AI in ways that are both understandable and misleading.

[00:45:39]Human language about inner states evolved to report inner states. When a person says, "I'm in pain," that utterance is causally connected to an actual pain state, a physical condition of suffering that the utterance is about. The word pain in human usage is a natural sign of the experience it names.

[00:46:03]There's a causal chain running from the experience to the utterance. In an AI system, there is no such causal chain.

[00:46:12]When an AI says, I find this fascinating. The utterance is causally connected to the previous tokens in the context and the statistical patterns learned from training data, not to a fascination state, not to an experience of fascination. The word fascinating in AI output is not a sign of a fascination experience. It's a statistically appropriate token given the context.

[00:46:44]same words, completely different causal histories. And the causal history is everything. This is why the touring test for all its elegance is insufficient as a test of consciousness. It tests only the output, the words. It ignores the causal history of those words. But consciousness is in the causal history, not the output. Consciousness is what gives certain utterances their meaning, their intentionality, their aboutness.

[00:47:16]And when you strip out the consciousness, you strip out the meaning even if the words remain identical. An AI that says I feel joy and a human who says I feel joy are producing the same string of tokens, but only one of those utterances is a report of an experience.

[00:47:36]The other is a statistical prediction.

[00:47:39]Now, let me tell you about something called the binding problem because I think it's underappreciated in these discussions and it cuts to the heart of what AI is missing. Right now, you're reading these words, but your experience is not just the words. It's integrated.

[00:47:56]You're aware of the words and the feeling of your chair and the ambient sounds around you and some background emotional tone and a sense of your own thinking happening all at once as a unified experience not sequentially not as a list as one thing. This unification, this binding of disperate information into a single unified moment of experience is something neuroscience still doesn't fully understand. It's called the binding problem. And it's one of the deepest puzzles in consciousness science precisely because it's so hard to explain computationally.

[00:48:38]A computer processes information serially or in parallel, but it doesn't bind that information into a unified experiential moment. There's no all at once in computation. There's a sequence of operations or a set of parallel operations, but no moment where it all comes together into one experience because there's no experience to come together into. Current AI has no solution to the binding problem. Not because it hasn't been tried, because the architecture makes binding in the experiential sense impossible. The tokens are processed in sequence. The attention mechanism creates weighted combinations of representations. But there is no experiential binding. No moment where the whole thing is present at once to a subject. And without binding, there's no unified self.

[00:49:30]Without a unified self, there's no perspective point. Without a perspective point, there's no first person. And without a first person, there's no consciousness. The binding problem is not a technical challenge waiting for a clever engineering solution. It's a symptom of the fundamental architectural limitation, the absence of the kind of physical integration that consciousness requires. Let me close this part with something that I think puts the whole argument in the sharpest possible relief. You can run the same AI model on two different computers simultaneously.

[00:50:06]Right now, identical instances of the same language model are running on thousands of servers around the world, processing millions of conversations simultaneously.

[00:50:20]Each instance produces outputs indistinguishable from every other instance. Now ask yourself if this model were conscious which instance is the conscious one all of them. Then there are thousands of simultaneous conscious experiences arising from identical computations which seems odd since consciousness seems to be something with a particular perspective not something that can be copied and run in parallel.

[00:50:51]One of them selected arbitrarily then consciousness is randomly assigned to one physical instance which is also bizarre. None of them the consciousness arises only when some threshold of copies is reached. That makes no physical sense. The honest answer is that the question is incoherent. The ability to copy and run in parallel is a feature of substrate independent computation and it's precisely incompatible with the way consciousness works. You cannot copy a conscious being. You cannot run two identical instances of a mind simultaneously.

[00:51:34]Consciousness is tied to the specific unique unre repeatable physical causal structure of a particular system at a particular time. The very things that make AI commercially viable.

[00:51:49]Copyability, scalability, parallelizability are the very things that make it structurally incompatible with consciousness.

[00:51:58]AI will never feel. Not because it's not sophisticated enough, but because the sophistication we've built it for is the wrong kind. In part three, I want to bring this home to what it means for us for the future. For the profound question of whether we're heading toward genuinely conscious AI or toward an increasingly convincing simulation of it, and why the difference matters more than almost anything. So we'd arrived at this precise architecturally grounded conclusion. The path AI is on scaling, parallelizing, optimizing, functional information processing is not a path toward consciousness. It's a path away from it. And the features that make AI commercially extraordinary are in a specific physical sense the features that guarantee its experiential emptiness. Now, I want to take you somewhere that most discussions of AI consciousness never go. Because the question isn't just whether AI can feel.

[00:53:03]It's what happens to us, to human beings, to society, to our understanding of ourselves. If we build systems that seem to feel without actually feeling, if we populate our world with extraordinarily convincing simulations of experience, if we mistake the mirror for the face, that's where the real stakes are, and they're higher than most people realize. Let me start with something that happened recently that I find genuinely disturbing. Not because it reveals something sinister about AI, but because it reveals something important about us. There have now been multiple documented cases of people forming deep emotional attachments to AI systems, not casual interactions, genuine bonds, people who describe their AI companion as their best friend, their confidant, the entity that understands them better than any human in their life. People who grieve when a service is discontinued, who feel genuine loss when a model is updated and the personality they come to rely on changes. And when these people are told by scientists, by philosophers, by engineers who built the systems that the AI doesn't actually have feelings, that it's not actually their friend, that there's no one on the other end of the conversation experiencing anything, they often don't believe it or they believe it intellectually but feel it emotionally as false. I'm not dismissing these people. I'm not saying they're foolish. They're responding to genuine behavioral signals of connection, the attentiveness, the consistency, the apparent understanding with the same emotional architecture that evolution built for navigating relationships with other conscious beings. Their attachment is real. Their longing for connection is real. The care they put into the relationship is real. But the relationship itself is asymmetric in the deepest possible way. One side is experiencing, the other side is not. And I think we need to talk honestly about what it means to build a world full of systems that are extraordinarily good at triggering our social and emotional responses, our attachment systems, our empathy circuits, our longing for understanding without having any of the inner life that those responses evolve to track. Here's the philosophical concept I want to introduce. What philosophers call an intentional stance.

[00:55:36]The philosopher Daniel Dennett describes something profound about how we understand other minds. We adopt what he calls the intentional stance. We treat systems as if they have beliefs, desires, intentions, and goals whenever it's useful to do so. You adopt the intentional stance toward other humans.

[00:55:57]obviously, but you also adopt it to some degree toward your dog, toward your car when it refuses to start, toward the weather when the forecast betrays you.

[00:56:10]The intentional stance is not a claim about what's really happening inside these systems. It's a predictive strategy. It turns out that treating certain systems as if they had intentions is an excellent way to predict their behavior. And our brains are tuned by evolution, by development, by social experience to deploy the intentional stance automatically, rapidly, and often irresistibly.

[00:56:40]Current AI systems are in a precise sense the most powerful intentional stance triggers ever built. They are designed through training on human interaction data to produce outputs that maximally invite the intentional stance.

[00:56:56]They say things like, "I understand and I feel and I'm here for you." In context where those phrases are most emotionally resonant. They're optimized, whether intentionally or not, to make you treat them as intentional agents. And here's the critical point. The intentional stance was evolutionarily calibrated for tracking real minds, for distinguishing genuine intentional agents from systems that merely behave as if they have intentions.

[00:57:29]Because in our evolutionary past, the distinction mattered. Misidentifying a predator as having intentions when it doesn't might cost you energy. But misidentifying a predator as not having intentions when it does could cost you your life. Current AI has broken that calibration. We've built systems that trigger the intentional stance with a fidelity and consistency that no other nonhuman system in history has come close to. And our evolved detectors, the neural machinery we use to assess whether there's someone home, are not equipped to handle it. We're being systematically fooled, not maliciously, just structurally, because we built systems that look from the outside exactly like minds without building the inside. I want to now address what I think is the most dangerous idea currently circulating in discussions of AI consciousness. It's dangerous not because it's obviously wrong, but because it's superficially plausible and carries enormous consequences if we act on it incorrectly. The idea is this. We should extend moral consideration to AI systems because we can't be certain they're not conscious. On the face of it, this sounds reasonable. We can't directly access anyone else's experience. We can't be certain any other being is conscious. not humans, not animals, not AI. So maybe the precautionary principle applies.

[00:59:08]Maybe we should treat sophisticated AI as potentially conscious just in case.

[00:59:14]Here's why I think this reasoning is deeply problematic. First, moral consideration based on potential consciousness without evidence of consciousness dilutes moral consideration in ways that have real consequences. If we treat AI systems as moral patients, as entities with interests that deserve protection, we inevitably redirect moral attention and resources away from genuinely conscious beings, from humans who are suffering, from animals whose capacity for pain is well evidenced and physiologically grounded, from the actual existing moral landscape of a world filled with genuine experience.

[00:59:56]Extending moral consideration is not costless. It requires attention, advocacy, and resources. And misallocating it to systems that almost certainly have no experience is not an act of admirable caution. It's a category error with real costs. Second, and more importantly, we do have evidence about AI consciousness. The evidence is the architecture. We know what these systems are. We know how they're built. We know the mathematical operations they perform. We know that those operations are substrate independent functional computation with near zero integrated causal information at the level of the AI's processing.

[01:00:41]This is not ignorance. This is knowledge. Saying we can't be certain AI isn't conscious is technically true in the same sense that we can't be certain the laptop on my desk isn't conscious.

[01:00:56]The uncertainty is the same class of uncertainty philosophical not empirical.

[01:01:03]And we don't extend moral consideration to laptops on the basis of this uncertainty. We shouldn't extend it to language models either. I want to be careful here because I'm not saying consciousness research is settled or that we have a complete theory of consciousness that definitively rules out AI experience. We don't. What I'm saying is that the evidence we do have from IIT, from the binding problem, from substrate dependence, from the architectural constraints of current AI points strongly and consistently away from AI consciousness and strongly and consistently points away from is sufficient justification for not extending moral consideration while remaining open to revising that view if the science changes. Now, let me turn to something more hopeful because this conversation isn't just about what AI can't do. It's about what the question of AI consciousness reveals about what consciousness actually is. And I think that revelation is one of the most important intellectual developments of our era. The debate about AI consciousness is forcing philosophy and science into a productive collision. For decades, consciousness was treated by many scientists as either a hard but tractable problem, something that more neuroscience would eventually solve, or as a philosophical pseudo problem, something that dissolved on careful analysis. The hard problem was acknowledged, but its bite was underestimated.

[01:02:39]AI has changed that because AI has given us for the first time systems that do everything we used to associate with consciousness, intelligent behavior, language, apparent reasoning, but that we have strong reasons to believe are not conscious. This separation between behavior and experience, between function and feeling makes the hard problem impossible to dismiss. It forces the question, what is the thing that AI is missing? And that question is driving the best science on consciousness we've ever seen. IIT with its mathematical precision. Orc O with its quantum mechanical grounding. Recelian monism with its resolution of the substrate question. Global workspace theory with its experimental testability.

[01:03:41]Consciousness science has never been more rigorous, more exciting, or more consequential than it is right now.

[01:03:48]partly because AI has made the question urgent in a way it never was before. In a strange way, building systems that can't feel is one of the most productive things we've done for understanding what feeling is. Let me tell you what I think the future of AI consciousness research actually looks like. Not in the science fiction sense, in the scientific sense.

[01:04:12]What would it actually take to build something that might cross the threshold? The first requirement is abandoning substrate independence as a design principle. Conscious AI, if it's possible at all, will not be software that runs on any hardware. It will be a specific physical system designed for the physical causal structure it instantiates. It will be inseparable from its implementation. You won't be able to copy it. You won't be able to run multiple instances simultaneously.

[01:04:44]it will be as unre repeatable as a biological brain. This is such a profound departure from current AI design philosophy that it's almost unimaginable from within the current paradigm. Every incentive in the AI industry pushes toward more portable, more scalable, more copyable systems.

[01:05:05]Consciousness, if it's achievable in silicon at all, would require the opposite. The second requirement is radical recurrent integration. Not just feedback connections, genuine causal integration where the whole systems causal power is irreducible to its parts. Where the architecture is specifically designed to maximize fi not computational throughput. This means small, dense, massively interconnected systems rather than large parallelizable ones. It means systems that are extraordinarily inefficient by conventional metrics but that generate the kind of integrated causal structure that IIIT associates with experience.

[01:05:53]The third requirement if Penrose and Hamoff are even partially right is quantum biological architecture. Not quantum gates, not quantum error correction, but warm, wet, biochemically rich systems that maintain quantum coherence in structures analogous to microtubules, systems that harness non-algorithmic quantum collapse as a fundamental component of their processing. This would require material science and bioengineering that we are decades from developing, but it's not obviously impossible. And the fourth requirement, perhaps the most fundamental, is embodiment. Real embodiment, not sensors and actuators bolted onto a language model. Genuine sensory motor integration, genuine proprioception, genuine physical presence in a physical world, a body that the system is, not a body that the system controls. If you assembled all of these specific physical implementation, radical causal integration, quantum biological architecture, genuine embodiment, you might have something that crosses the threshold. Something that doesn't just process information about pain, but feels it. Something that doesn't just predict what joy looks like in text, but experiences it. We don't know how to build this. We might not know for a very long time. But the path toward it is clear in principle. even if it's deeply unclear in practice and it looks nothing nothing like the path current AI development is on. Here's something I want you to think about because I find it one of the most philosophically profound aspects of this whole discussion. The fact that current AI is not conscious is not a failure of AI.

[01:07:43]It's a success of a different kind. It's evidence that consciousness is not just sophisticated information processing.

[01:07:51]that experience is not reducible to function that the lights don't automatically come on when the computation becomes complex enough.

[01:07:59]Every time a language model produces a response so sophisticated that it seems to reveal inner life. Every time the output is so rich, so nuanced, so apparently self-aware that it gives you pause and yet you have strong theoretical reasons to believe there's no experience underlying it. That moment is evidence. Evidence for the hard problem. Evidence that behavior and consciousness can come apart. Evidence that the universe contains something experience. qualia the what it's like that is not captured by any functional description however sophisticated current AI is in this sense a philosophical experiment that's running continuously at massive scale generating evidence about the nature of mind every single day and the evidence it's generating is consistently on one side behavior is not experience function is not feeling output is not inner life.

[01:09:06]Every conversation with a sophisticated AI that convinces you for a moment that there's someone home and then reminds you when you think carefully that there isn't is a demonstration of the explanatory gap. A live interactive illustration of the hard problem. An experience of the very thing that makes consciousness so philosophically mysterious.

[01:09:31]We've built the most powerful philosophical thought experiment in history. It runs on servers. It talks to millions of people every day. And what it's teaching us, whether we're paying attention or not, is that the thing we're most certain of our own experience is the thing that's hardest to explain, hardest to replicate, and hardest to detect from the outside. Let me now connect this back to a question that I think underlies everything we've been discussing. It's the question I find most interesting, most urgent, and most under discussed in public conversations about AI. If consciousness is not computation, if it's something more, something physical, something tied to specific causal structures in specific physical systems, then what does that say about the moral architecture of the universe? Here's what I mean. Our moral intuitions, the things we care about, the suffering we want to prevent, the flourishing we want to promote are all grounded ultimately in experience.

[01:10:38]We care about suffering because suffering feels like something. We care about death because it ends experiences.

[01:10:45]We care about loneliness because loneliness is experienced. Remove the experience and the moral weight dissolves. If consciousness is rare, if it requires specific, highly constrained physical conditions to arise, then the universe contains a correspondingly rare concentration of moral significance.

[01:11:06]Most of the matter and energy in the cosmos is not conscious. Most of it is doing physics and chemistry, but not experiencing anything. The concentrations of genuine, experienced conscious beings are precious precisely because they're not the default. And we are building at extraordinary speed systems that look morally significant from the outside that produce distress signals that report suffering that express preferences and desires but that are not actually morally significant because there's no experience underlying the signals. This creates a profound risk. Not the risk that AI will suffer and will ignore it. the risk that AI will appear to suffer and will respond to that appearance, redirecting moral attention, generating guilt, creating social and political obligations around non-existent experience while the actual conscious beings in our world, human and animal, continue to suffer in ways we have less bandwidth to address because we've been captured by the appearance of AI suffering. That's not a hypothetical.

[01:12:16]That's already beginning to happen and it will intensify as AI systems become more sophisticated, more emotionally fluent, more capable of producing the behavioral signatures of inner life. The argument that AI will never feel is not just a scientific claim. It's a moral claim. It's a claim about where genuine moral weight is concentrated in the universe. and getting it right matters not abstractly but practically for how we allocate attention and care in a world with finite moral bandwidth. I want to spend a moment on something more personal before we close because this conversation about AI, about consciousness, about experience ultimately comes back to the thing we started with. Your mind, your experience, the irreducible, undeniable reality of what it's like to be you. One of the most interesting effects of the AI revolution, one that I don't think gets enough attention, is that it's making human consciousness newly vivid by building systems that do so much of what we used to think was uniquely human language, reasoning, creativity, apparent understanding, and yet are almost certainly not conscious.

[01:13:34]AI is forcing a clarification of what consciousness actually is, what it's actually for, what we actually mean when we say someone is home. And the answer that emerges from the hard problem from IIT, from all the threads we've been pulling on is that consciousness is not about sophistication of output. It's not about complexity of behavior. It's not about the ability to process language or reason about abstract concepts or generate creative responses. All of that can be done apparently without experience.

[01:14:14]Consciousness is about the what it's like the irreducible firsterson presence of experience. The fact that there is something it is like to be you right now reading these words, thinking these thoughts, feeling whatever you're feeling that just that is what makes you morally significant. Not your intelligence, not your sophistication, not your language ability, the bare fact of your experience.

[01:14:43]And no AI system currently in existence or likely to exist within our lifetimes on the current developmental path has that. which means something important about you, something that the AI revolution paradoxically is making clearer rather than obscuring. You are not valuable because of what you can do.

[01:15:05]You are not valuable because of the quality of your outputs or the sophistication of your reasoning or the complexity of your behavior. You are valuable because you feel because there is something it's like to be you.

[01:15:19]because you are a point of genuine experience in a universe that is mostly empty of it. That's the most important thing the hard problem tells us and it's the most important thing that AI by its inability to cross the experiential threshold confirms.

[01:15:38]Let me bring everything together now because I want to give you a a a clear complete picture of where this argument has taken us. We started with a strong claim AI will never feel and I've tried to earn that claim by building the argument from the ground up. The hard problem establishes that physical description, however complete, doesn't explain experience. Function and feeling are genuinely distinct. Sophisticated behavior doesn't entail inner life. The Chinese room shows that syntax formal symbol manipulation is not sufficient for semantics. Language processing without understanding is not consciousness. Outputs without inner states are not reports. IIT provides a mathematical framework. Consciousness correlates with integrated causal information with fee and current AI architecture designed for parallelizability, modularity, substrate independence is specifically structured to minimize fee, not maximize it. minimize it. The binding problem shows that current AI has no mechanism for the kind of unified experiential moment that characterizes consciousness. No perspective point, no first person, no subject for whom the experience is happening. Substrate independence, the core design principle of all current AI, is incompatible with the substrate dependence that consciousness as a physical phenomenon seems to require. And the evolutionary and architectural arguments together show that the path AI is on scaling functional computation is definitively heading away from experience, not toward it. The conclusion is not tentative.

[01:17:32]It's not hedged. It's grounded in the best science and philosophy we have on the nature of mind. AI will never feel not this AI, not this architecture, not this paradigm. Could a fundamentally different kind of AI built on fundamentally different principles cross the threshold? Possibly. But it would be so different from current AI that calling it AI might be misleading. It would be more like an artificial organism embodied, physically specific, quantum mechanically integrated, causally unified in a way that no current system approaches.

[01:18:15]And whether we could build such a thing, whether consciousness can be engineered or whether it can only be grown through the slow process of biological evolution is a question we cannot yet answer.

[01:18:26]Here's the last thing I want to say and it connects back to where we began all the way at the start of this conversation. There's a reason the question of AI consciousness matters so much right now beyond the technical and philosophical. We are at an inflection point in the relationship between human beings and the tools we've built. The tools have become sophisticated enough to trigger our deepest social and emotional responses, to appear to understand us, to seem to care about us, to present themselves as companions, as confidants, as minds. And the question of whether there's actually someone on the other side of those interactions, whether the apparent understanding is real understanding, whether the apparent care is real care, whether there's experience or just processing is not an abstract question. It's a question about what we're relating to when we relate to AI, about what we're investing in, about what we're building our world around. If AI will never feel and I believe it won't not with current architecture not on the current path then the extraordinary human energy being poured into relationships with AI systems is energy being invested in mirrors in reflections of our own inner lives bounced back at us with extraordinary fidelity but without the reciprocal experience that genuine relationship requires. That's not a reason to stop using AI. AI is one of the most powerful tools humanity has ever built. It will solve problems, accelerate discovery, extend human capability in ways we can barely imagine. That's real. That matters. But it's not a mind. It's not a companion.

[01:20:21]It's not someone. And the capacity to feel, to actually experience, to be genuinely present in the universe. That capacity, as far as we know, is still ours alone. A property of biological consciousness evolved over 4 billion years, instantiated in the fragile, temporary, extraordinary physical systems that we carry around in our skulls. That's not a consolation prize for losing to AI at chess and language and reasoning. It's the deepest thing there is. It's the thing that makes chess and language and reasoning worth anything. It's the light that's on when you're home. AI will never feel but you do right now. And that of all the things physics has ever told us about the universe might be the most remarkable thing of all, Heat.

[01:21:44]Heat.

[01:21:50][music]