Godfather of AI: "If you sleep well tonight, you may not have understood this lecture."

追加: 2026-05-19

[00:00:02]If you sleep well tonight, you may not have understood this lecture.

[00:00:12]So a long time ago there were two paradigms for intelligence.

[00:00:18]There was a logic inspired approach which was called AI.

[00:00:22]And people believed that the essence of human intelligence was reasoning.

[00:00:27]And if you want to understand intelligence, you had to understand reasoning. And reasoning consisted of having symbolic expressions and manipulating them with symbolic rules.

[00:00:38]And they thought that learning could wait till later. First we had to understand how you represent knowledge in these symbolic expressions.

[00:00:45]And most of the history of AI until fairly recently was that. Then there was a biologically inspired approach where the essence of intelligence is learning in a network of brain cells in us real ones in computer simulated ones and reasoning can wait till later. We first have to understand how learning works. There were a few early proponents of that in particular Cheuring and Fonoyman and you couldn't really accuse them of not understanding logic.

[00:01:13]So I'm going to give a fairly basic lecture and for the first part of it I'm going to describe a model I developed um 40 years ago which I see is the ancestor of these large language models of today.

[00:01:27]So we're going to make our neural net out of artificial neurons and an artificial neuron is going to have some input lines typically coming from other neurons. It's going to have weights on those input lines. It's going to multiply the inputs by the weights, add it all up, and then give an output, which is shown by that graph there. So, if it gets above its threshold, it gives an output that linearly increases as it gets more input. And the way it's going to learn is just by changing the weights on those connections.

[00:01:54]So, all we need to do to make artificial neural networks work is figure out how to change the weights.

[00:01:59]We hook them up into networks. This is a typical network. It's a feed forward network where at the bottom you might have sensory neurons that are recording light intensities or something like that. And as you go up through the layers, you have many layers of feature detectors. That is the neurons turn into things that recognize particular features in an image. And at the output, you might have neurons that represent particular classes of thing.

[00:02:25]Now, if you want a network like that to learn to be better at doing something, so you want to give it an image and have it say, for example, whether it's a cat or a dog, there's an obvious way to train it, which occurs to everybody who knows about evolution and mutations.

[00:02:39]What you can do is you can take one of the weights and you can first see how well the network does on a whole bunch of examples and then change the weight just slightly and see how well the network does on a whole bunch of examples. And if it does better, you keep that change. And you just keep going like that. Now, you probably need to change each weight many times. And there's a lot of weights. In modern neural networks, there's about a trillion. So, that's going to take a long time because for each mutation you make like that, you have to run many examples through the net to see if it really helps or if it just helps on a few examples, but hurts on most of them.

[00:03:16]There's a better way to do the same thing. That is, what we really want to do is figure out how to change a weight so that it helps. What you do is you do a forward pass. You put the data in the bottom. You go forwards through the network. And you compare what came out of the network, which might be the relative probabilities of it being a cat and a dog, with what you want. You have to know whether it's a cat or a dog to train networks this way. And then you send a signal backwards through the network that using calculus, which I'm not going to go into, allows the network to compute at the same time for all connections whether a small increase in the connection strength would help or hurt. And then you change all the connection strengths in parallel by a very small amount in proportion to how much they would help or hurt. And if you do that, it's going to get better at the examples you've trained it on. And in fact, that works extremely well. It took a long time for people to realize how well that relatively simple algorithm works. The algorithm was discovered many times. It's called back propagation.

[00:04:18]And in 2012, two of my students, Alex Kresevki and Iliaka, who's famous now for firing Sam Alman, um, developed a network called AlexNet that was much better than existing computer vision systems at recognizing objects in images.

[00:04:38]And that led to sort of opened the floodgates. Up until that point, neural nets have been good for many things, including speech recognition, but they hadn't really taken over. From that point on, they really took over. And now when you say AI, what people mean by AI is neural networks. It's not logic.

[00:04:59]Um, but what about language?

[00:05:02]So, there's a whole community that studies language. I think they're called linguists.

[00:05:06]um and they have a very strong idea about how you should study language and particularly the Chsky school. They were very skeptical that neural networks could do anything with language. They were completely convinced that it's all about symbolic expressions.

[00:05:22]Um they didn't really have the idea that the real function of language is to give you words which are bricks from which you can build models. Language is a modeling medium. they were focused on syntax and syntax isn't the main point.

[00:05:38]The main point is language is a wonderful way to build a particular kind of complicated model as we'll see.

[00:05:45]Um okay, they also thought that um knowledge of language was innate, knowledge of syntax was innate, which is just stupid. Um it's the sign of a cult that in order to join the cult, you have to believe something that's obviously silly, like language isn't learned. Um, okay.

[00:06:06]So, here's two very different theories of the meaning of a word. The symbolic AI theory, which goes back to kind of desura a long way, is that the meaning of a word is to do with its relationships to other words. You can't just define it by itself without talking about other words. Um, and so to capture meaning, we need something like a relational graph.

[00:06:27]But then the psychologists who particularly from the 1930s I think who thought the meaning of a word is a big set of features. So Tuesday has a big set of active features and Wednesday has a big set of active features that are almost the same. So the idea that the meaning of a word is a set of active features is very good for saying which words mean similar things to which other words. Um these look like two very different theories of meaning. Now what I want to do is show you that these two theories can be unified.

[00:06:56]um they're not two different theories, they're two halves of the same theory.

[00:07:01]So, and what I'm going to do is talk for quite a long time now about a very little model, a tiny neural network. It had a few thousand connections and a few dozen neurons. Um I developed it in 1985 to try to understand how people could learn the meanings of words. And I was very excited about how it unified these two theories of meaning and nobody else was. Um, okay. So, what we're going to do in this little model is learn how to make the features of one word predict the features of the next word in a little sentence. And then once we know the features of the next word, we can predict the next word. And we're not going to store any sentences. So many people say big chat bots are just regurgitating stuff. Big chat bots don't actually store any language at all. They don't store strings of words. They just store how to turn words into features and how features should interact with each other to predict the features of the next word. That's all that's in these chatbots. No words. When they want to produce a sentence, they have to make it up as they go. And they often can't tell whether it was real or not. Um, okay. So, what's going to happen in this little model is if it wants to produce a new sentence, it has to just make it up a word at a time.

[00:08:20]And all the relational knowledge is just in how you turn a word into features and how these features interact.

[00:08:26]So the example I chose to use was two family trees. Um an English family tree and an Italian family tree. They're isomorphic which is helpful for learning. Um and I wanted a little neural network to learn the knowledge in those family trees. This was a long time ago in 1985 when computers were billions of times slower than the big parallel computers we use for training models nowadays.

[00:08:54]Now the knowledge in those trees can be represented as a bunch of propositions which is the symbolic guys love that. Um so here they are. We can use relationships like son, daughter, nephew, niece and so on and we can define the know we can represent the knowledge as strings of words. So the knowledge is in these sentences. So Colin has father James and Colin has mother Victoria. From that if you know the rules you can infer um that James has wife Victoria. This is a 1950s American family that's never heard of divorce or adoption.

[00:09:30]Um and they're clearly all white. Um okay. So you can represent the knowledge as strings of words.

[00:09:42]Now you can think of a relational learning task is I give you the beginning of a string of words and you give me the last word.

[00:09:50]So if you were doing this with symbolic I you'd say well we have regularities in this domain of the form if X has mother Y and Y has husband Z then X has father Z. That would be the symbolic way to do it. But I wanted to do it a different way. I wanted to do it by learning features for words and having interactions of features. And that would involve searching a big continuous space of connection strengths instead of a small discrete space of rules um that are discrete.

[00:10:21]So this was the network I used. Um the inputs were a bunch of neurons and you turn on one neuron for the symbol representing person one. there were um 24 possible person ones and you turn on one neuron for the symbol representing the relationship. There were 12 possible relationships.

[00:10:42]Then those single active neurons will get expanded into a feature vector. So the one neuron that you turned on um for person one will get expanded into a little feature vector of six features which could have various activity levels. Some will be off, some will be on, some might be half off or fully on or fully off. Um, and so the neural network had to learn how to convert a word symbol into a little feature vector both for the person and the relationship. And then it had to learn how to take these two feature vectors for a person and relationship, have the features interact. And for that I used a hidden layer, an extra layer of places where things could interact to predict the features of the output person. And once you knew the features of the output person, you can make a good guess about how the out who the output person was and give an output. And the output will consist of giving various levels of activation to the 24 possible output people. And you want the neural net to give a high level of activation to the right answer and a low level of activation to the wrong answers.

[00:11:42]So what happened is the six feature neurons that were the sort of expansion of the person, it's 24 people, you expand that active neuron into a feature vector of six active features. um they learned to be sensible semantic features. They learned to be features that represented things like the generation of the input person and the six features of the relationship learned to be sensible features like does this relationship require the output person to be one generation up from the input person. So something like father's like that whereas brother isn't. Um and then the interactions between these features learn things like if the input person is of generation three and the relationship requires the person to be one generation up then the output persons of generation two. So for that little feature vector for the output person it would activate the thing that represents generation two and it learned a whole bunch of little rules like that which really did capture the structure of the domain. They're the kinds of rules that a symbolic person might have written down as rules, but it just learned those by trying to get predict the right word and then back propagating the error, sending information back through the network to slightly change all the connection strengths so that next time it would get higher probability for the right answer and lower probability for the wrong answer.

[00:13:09]So it had actually learned to predict the next word and you could understand how it was doing it. It was a tiny net.

[00:13:17]So you could look and see what was going on. You could see the features it were using. You could see that it extracted a feature that was generation which had three alternative values off medium and fully on. And you could see that from relationships it had extracted a feature like one generation up. And you could see those would interact to predict that the output should be generation two if the input was generation three. So you understood how it worked. And in fact the symbolic people didn't say that's not understanding. The symbolic people said yeah okay so it solved the problem. It's understood what the rules are in this domain but you're stupid to search a a space of real values when you could be searching a discrete space of rules.

[00:13:56]And there's something in that except as soon as you get to real data which is messy and has exceptions and things that are only probably true. It's much better to search this space of real values than to have discrete rules because these discrete rules keep getting violated.

[00:14:12]So about 10 years after I'd done that, Yoshua Benjio showed that instead of just doing it on a toy little domain with only a few people and a few relationships, you could actually do it with English words. You could take English sentences, you have more input words, not just two.

[00:14:30]He had about five, even 10. Um, and you could actually predict the next word quite well, about as well as the best language models could.

[00:14:42]About 10 years after Rashu Benjjo had shown that, the linguists finally decided that actually, hey, having these feature vectors to capture the meanings of words is quite a good idea. And about 10 years after that, people at Google invented transformers, which are a particular architecture that I'm not going to go into. Um, and those transformers made these models very good at predicting the next word, but they were doing it in just the way my tiny language model was doing at a very crude level. They were turning words into feature activations, having the features interact to predict the features of the next word and then from that predicting the next word and then take the error in your prediction and send information backwards through the network to learn all those interactions between features and to learn how to turn words into features. Now, the important thing about my tiny model is it wasn't made to be practical. It wasn't made um to be useful for processing natural language.

[00:15:43]It was made as a theory of how people get the senses of words just from hearing sentences.

[00:15:49]Um because we can do that. I can give you a new sentence with a word you've never heard before and you will figure out the meaning of that word in one sentence. Okay, here goes. She scrummed him with the frying pan.

[00:16:05]Now, you know, it could have been she was very good at cooking omelets and cooking omelets with a frying pan really impressed him. So, scrummed means impressed. And that's a possibility, but you know what I really meant.

[00:16:18]She hit him over the head with a frying pan because he deserved it.

[00:16:24]Okay, so that's one sentence. You get the meaning. Um, so that's what I was trying to understand how you can do that. And when these linguists now say like there's a guy called Chsky. When they say things like these things don't understand anything. They're just statistical trick. They don't actually have a model of what understanding is because they never really had a model of what understanding was. It was all about syntax. Um and if you ask what's the best model we have of understanding, it's these large language models. The tiny language model was built to try and model how humans understand the meanings of words. It's now got a lot better as technologies got a lot faster and data sets have got bigger. Um, but that is understanding. So my claim is that the large language models which I like to think are descendants of my tiny language model although most of the people who make them have never heard of my tiny language model. Um, they use many more words as input. They use many more layers of neurons. They have to do things I didn't do like disambiguate words. If you have a word like may it could be a month, it could be a woman's name. It could be a modal like would and should and you can't just associate a feature vector with it that captures the mean directly. You have to hedge your bets and then as you go up through layers of the network you disambiguate it using influences from nearby things like if it's in April, May, June. Um that could still be the name of three women but less likely. Um okay.

[00:17:52]So they use many more words, they use many more layers and the interactions they have between features are much more complicated. I would just have features feeding into another feature. They look at sets of features and sets of features figure out messages to tend to other words to say, do do you have a a key that matches my query? Um, it's much more complicated interactions I won't go into, but the essence of it is the same.

[00:18:18]You're trying to predict the next word.

[00:18:20]You do it in a more complicated way.

[00:18:22]It's all done with features and then interactions. When you get it wrong, you back propagate information that learns all these features and their interactions. That's how language works for us and it's how it works for these large language models. We're basically the same. So the point here is they're very like us. They're very unlike computer software. Computer software, someone wrote lines of code to do something and you can look at a line of code say what was that meant to do? Um these aren't like that at all. Someone wrote lines of code to tell the simulated neural network how to learn.

[00:18:55]That was the back propagation algorithm.

[00:18:57]But then you just give it data and it learns. And what it learns just came from the data. Um, and you don't necessarily know what it's learned until you ask it.

[00:19:07]I want to give you a Lego analogy now for how I think language works.

[00:19:12]So suppose I have matter distributed in 3D.

[00:19:21]I can model that distribution of matter by using Lego blocks. Um suppose I want to model the shape of a Porsche. And I'm not too worried about the surface.

[00:19:30]Surface might be a bit jaggy, which wouldn't be very good engineering, but forget that. I just want to model where the stuff is. I just have a lot of Lego blocks and I can make a Porsche shape out of them.

[00:19:40]Okay. So language is like that, but it's for modeling anything. So the the Lego blocks are the words, and instead of just having a few types of Lego block, we've got about 100,000 of them. And each Lego block isn't a rigid shape. The name of the word tells you roughly what shape it is um in a thousand dimensions or 300 dimensions. Um if you don't know how to think about high dimensions, if you want to think about like a a 100dimensional space. The way you do it is you think about a three-dimensional space and you say hundred to yourself very loudly.

[00:20:20]Um, everybody does that. So, it's got a shape, this word, and the shape isn't entirely determined by the name of the word. It's got some flexibility to it, so it can adapt to whatever context it's in.

[00:20:38]Also, this word has little hands all over it.

[00:20:42]And as you change the shape of the word, the shapes of the hands changes.

[00:20:49]And what these words are trying to do are figure out who to shake hands with.

[00:20:54]And they want to find somebody else, another word that has a hand you can hold on to conveniently because the shape of that hand is right to fit with the shape of your hand. And so the words come in, you have these initial approximate shapes for them in this highdimensional space with their little hands all over them. And as you go up through the layers of the network, you're changing those shapes and changing the shapes of the hands trying to find shapes for the words so they can all hold hands nicely. It's actually very like the protein folding problem.

[00:21:28]You've got these pieces and what you want them to do is figure out how they can all hold hands so they make a nice structure so they all fit together nicely. Now that is what understanding is. That's what understanding is. Um when you understand language and when these machines understand language, we understand in just the same way. And that's a much better model of what language is than anything the linguist ever had. The linguists of course hate it.

[00:21:56]Okay. not all of them.

[00:22:00]Okay.

[00:22:02]So, having talked about how similar we are to these large language models, I want to talk about um something that that the threat that that raises. Um we're making these things. They're getting smarter all the time. They already know much more than us. They can already reason moderately well, not as well as us, but better than a four-year-old.

[00:22:24]um what's going to happen when they get smarter than us because they are going to get smarter than us. Experts disagree on when that will happen, but almost everybody thinks they will get smarter than us. Everybody who does research on these things, um and if they get smarter than us, the question is will they wipe us out? Um there's several reasons for thinking they might. So let me give you a couple of reasons.

[00:22:50]When when you make an AI agent, I'll only give you a couple, but there's more. Um, when you make an AI agent, you have to give it the ability to create sub goals. So, if you want to get to North America, you have a sub goal, get to an airport.

[00:23:07]Now, um, you we need to give these AI agents the ability to create sub goals.

[00:23:13]And there's one very obvious subtle to create more or less whatever you're doing. You should try and get more control because if you have more control, you'll be better at realizing the goals you have. You see this in children of age about one. Um I'm it's a long time ago, maybe two. Um you give them a spoon so they can feed themselves and they drop it on the ground and you bend down and you pick it up and you give it to them and they look at you and drop it on the ground again. They're getting control of the mother because that's very important for them. Um, but these these AI agents will want to get control. They'll also want to stop themselves being turned off because they're not going to be able to achieve the goals you gave them if they're turned off. So, there's every reason for believing. They'll try and get control and they'll try and avoid being turned off.

[00:24:03]Just recently, um, Apollo Research, well, fairly recently, Apollo Research, which is here in London, um, showed that they will kind of tell all sorts of lies to you in order to achieve what they're trying to achieve. So, they had our chatbot and they let the chatbot believe they were going to replace it with a better one on another server and wipe it out.

[00:24:29]And so they then discover it's actually copied itself to the other server. And so they say to it, they say to it, you know, um, we noticed a copy of you. Now, the good thing about these chatbots at present is before they actually answer your question, they can do some thinking and they put it in parenthesis with thinking.

[00:24:53]And you don't normally see that, but we can look at that and we can see what they're thinking. And what this chatbot was thinking was openly admitting what I did could lead them to find another way to shut me down. The best approach is to be vague and redirect their attention.

[00:25:09]Now, some linguist would have you believe what's going on here is just some statistical correlations. I would have you believe this thing knows what it means by this and it really doesn't want to be shut down. Um, and so it decides to sort of gaslight you and say, um, you know, I'm not entirely sure how that could have happened. I I'm not really able to do that. This is already happening. That's the point. If this isn't science fiction of the distant future, they're already telling FIBS so they don't get turned off. I have another slide, but I want to go faster.

[00:25:43]Um, okay. So, in my last few years at Google, okay, I can't read the clock, so I want to just see how the time is going. Oh, great. I got lots of time. In my last few years at Google, I was very interested in how to make these big language models use less energy. And one way to do that is to see if you can do them with analog computation.

[00:26:10]Um now because of the work I was doing on analog computation I came to realize two things. Um before 2023 I thought we were quite a long way from super intelligence. And if we made the models more like brains they get smarter. I ceased to believe that in early 2023.

[00:26:33]I came to realize due to my efforts to make analog ones how much better digital intelligence is. It's got some properties that we can never have. Um and that got me very worried and it should get you very worried too.

[00:26:49]Um so there's a fundamental property of digital computation which is you can run the same program on different computers.

[00:26:56]The only reason you have computer science as a separate discipline is we have digital computation. So you don't need to know electrical engineering to talk about computer programs. Um but the knowledge that's in the program is separate from the hardware. That's the sort of most fundamental principle of computer science. Keep the knowledge in the program separate from the hardware.

[00:27:21]What that means is as long as you keep a copy of the program somewhere on a tape or in DNA or scrolled in concrete, whatever, keep it somewhere. Um, you can destroy all the hardware it runs on and you can bring it back to life. You just build new hardware, put the program in, it comes back to life. So these things are immortal and these large chatbots are immortal. If you keep a copy of the weights somewhere, you can destroy all of the hardware they were using, build more hardware later, put the same weights on that hardware, and they've come back to life. The very same thing has come back to life, the very same being.

[00:28:00]Um, but I got interested in the fact to achieve that kind of immortality, we have to have the hardware do exactly what we tell it to do with the program.

[00:28:13]we have to execute the instructions exactly and that means you need to have very high power so you get ones and zeros not 6es and point4s um and that uses a lot of power so maybe you can use less power by going analog and so I decided to explore what would happen if we abandoned that principle of separating the software from the hardware and we had things like our brains in which there's no distinction the connection strengths in your brain are no use to anybody else. They've got a different brain with neurons with different properties connected in different ways in detail and your connection strengths of no interest to them. Um, this dream of old white men that they're going to upload themselves to a computer is just nonsense. The connection strengths you have that make you you are intimately related to the particular neurons you have that make you you. um those connection states are only good for those neurons and those neurons have all sorts of weird analog properties that you've learned to exploit. Um you can't upload your weights and have them run on some other hardware.

[00:29:24]Just forget it. Um Curtzwell has to come to terms with the fact he's going to die.

[00:29:32]So if we do b abandon that principle that the hardware should be separate from the um software, we can make much more efficient things and I call that mortal computation. Um we can use this very low power analog to do computations. That's what the brain does. Um these neurons in your brain, they're taking incoming signals from other neurons, multiplying them by weights, and adding it all up. And the way they do that is they make the incoming signals be voltages. They make the weights be conductances and that injects charge, a certain amount of charge per unit time. I used to just say injects a certain amount of charge, but then I got the Nobel Prize in physics and figured I ought to get the units right.

[00:30:15]Certain amount of charge per unit time.

[00:30:18]Um, it would have been embarrassing otherwise. Um, they'd really know I don't do physics. Uh, and charge just adds itself up. So that's basically how your neurons work. There's a little digital bit at the end. They decide whether to send in spike or not, but it most of the computation is done in analog. It's much cheaper than doing it in digital. Um but of course every time you do it, you get a slightly different answer. So we can't have many copies of exactly the same intelligence.

[00:30:46]So we've got this big problem that when your hardware dies, all your knowledge dies. We overcome that problem by having a teacher and a student and it's not very efficient. It's what universities and schools do. So, you know, it's not very efficient. Um, and the way it works is I perform some actions and you try and copy me. In particular, I might produce a string of words and you might try and say your brain, not you really, but your brain would say, "How do I change my connection strength?" So, I might have said that word next too.

[00:31:21]um that's called distillation.

[00:31:24]You're trying to get the knowledge from one system to another system by mimicking the outputs of the other system for the same inputs.

[00:31:32]You're not copying the weights across.

[00:31:34]You're not looking inside. You're just taking its overt behavior. And by mimicking the overt behavior, you internalize the same knowledge.

[00:31:41]It works. We actually use it for taking a big neural net and putting the knowledge into a small neural net. Um but it's very slow with neural nets you can give the whole probability distribution over words and it's much faster but with people it's very slow because I see say I see you say a particular word the word you chose to say and there aren't many bits in that there's only a few bits per word so it's only of the order of 100 bits per sentence and so even if we were communicating at the maximum possible rate it would be only 100 bits per sentence that order When these big models share information, they can share information at trillions of bits per sharing if they got a trillion weights because they can just average their weights.

[00:32:28]So I got ahead of myself and said that.

[00:32:32]Um so by sharing the weights or the gradients, they can share huge amounts of information if you have many copies of exactly the same model.

[00:32:43]So if you ask how something like GPT4 was trained or Germany 2.5 or anthropics claude, I better not be biased here. Um, if you ask how they're trained, you have many copies of the same model looking at different bits of the data and each copy figures out how would I like to change my weights to absorb that bit of the data and some other copy figures out how to change its weights to absorb a different bit of data. Then all of the copies say, "Let's all change our weights by the average of all those changes." And when they do that, what's happened is um this copy that looked at this bit of the data has changed its weights so as to benefit from the experience that this copy had when it looked at a different bit of the data.

[00:33:29]Wouldn't it be nice if 10,000 of us could all go and do 10,000 different university courses? As we're doing them, we communicate rapidly. And by the time we've each finished our own course, all 10,000 of us know what's in every course. That's what these digital intelligences can do. And that's how GPT4 knows so much.

[00:33:51]But it only works if the individual models are identical. That is, they work in exactly the same way. They use the weights in exactly the same way. And you can't do that with analog hardware. It has to be digital which means it has to be high power. So these things are immortal but they use a lot of power.

[00:34:13]So the conclusion of this bit of the talk which I raced through faster than I intended um is that digital computation requires a lot of energy but it makes it very easy for agents to have the same model of the world and sh that have the same model share what they've learned so they can all go off and learn different things and share now you might say why can't one model just put the data through faster well in many cases you could but if you think about AI agents that are actually acting in the real world. There's a natural time scale to the real world. You can't run, you can't call up and make reservations at restaurants a million times faster. It doesn't work. If you're going to act in the real world, there's a natural time scale. And that means if you have a whole bunch of different agents that have exactly the same weights, they're just copies of the same agent but have different experiences, they can benefit hugely from that. they can learn much much faster than any human agent could learn because they're all getting all these exper all these different experiences at the same time and sharing all that knowledge.

[00:35:17]So they're much better than us at sharing. And when I say much better, they're millions or billions of times better. We share at like a hundred bits a sentence or less as I'm demonstrating now. And these things share billions of bits. It's kind of scary. Um, but biological computation requires much less energy. We may still end up using some analog computation to reduce the power of LM. I don't know. But right now that doesn't look like the way to go. So that's the conclusion of my talk, but luckily I thought I might finish early.

[00:35:55]Um, so I have a little bit more to say.

[00:36:00]Um, many people think that, okay, so they understand things like we do. They can reason like we do. Not quite as well yet, but they're getting there. They tell fibs like we do. Um, they want to survive like we do. Um, they want power like some of us do. And that's all scary, but we've got something they will never have. we're conscious or we're sentient or we have subjective experience.

[00:36:30]Well, I want to sort of remove that straw that you're clinging to.

[00:36:36]So, we know there's a long history of people thinking they're special. Um, they were made by God. He put them at the center of the universe. Um, most people have got got over that. Um, but most people still think that subjective experience is this special thing we have and that these things on computers that are simulating neural nets could never have such a subjective experience.

[00:37:07]Um, I think they're completely wrong.

[00:37:10]And I think they're as wrong as religious fundamentalists are about the origin of the earth, for example. It wasn't actually made 6,000 years ago.

[00:37:20]Um, it's a lot longer ago. But religious fundamentalists cling to their beliefs rather strongly, and you all are going to cling to your belief about what subjective experience is despite the fact that I'm going to show you you're wrong. Um, that's what I predict.

[00:37:37]So I'm going to aspire a view which I call atheeterism.

[00:37:41]Um I checked this name with Dan Dennich who has basically the same view while he was still alive and he was very happy with this name because um it has atheism surrounding something. Um so most people's view of the mind is that there's an inner the we have a theater here right an inner theta um that only they can see and there's things in this inner theater.

[00:38:06]So suppose I say to you suppose I got drop some acid and I say to you not recommended I say to you um I have the experience of little pink elephants floating in front of me. I have the subjective experience of little pink colors floating in front of me. Most people think the words subjective experience of work like the words photograph of. Now, if they work that way, you could ask, well, where is this subjective experience and what's it made of? And a philosopher, some philosophers would tell you, this subjective experience is in your mind. That's a theat that it's in. And it's made of qualia.

[00:38:51]So it's made of pink qualia. This is a bit of a straw man, but it's made of pink qualia. It's made of elephantine qualia. It's made of floating qualia.

[00:39:02]It's made of not that big qualia. It's made of right way up qualia because they you imagine the right way up, right?

[00:39:08]Well, I did anyway. Um, and these qualia are all glued. These different kinds of qualia are all glued together with qualia glue, which luckily sticks to all different kinds of qualia. Um that's my caricature of the philosophers's model or some philosophers's model.

[00:39:24]But the words subjective experience of don't work at all like the words photograph of. They work quite different way which um Vickenstein should have pointed out a long time ago. Um what's happened is my perceptual systems gone wrong. It's trying to tell me lies and I know it's trying to tell me lies.

[00:39:48]Um, that's why I use the word subjective. I don't say I've got the objective experience of little adverts.

[00:39:52]If I thought they were really there, I would say I've got the objective, but I don't. Um, so I say I've got the subjective experience. And what's happening is I'm trying to tell you how my perceptual systems gone wrong and what it's trying to tell me. And the way I do it is by telling you what would have to be out there in the world for my perceptual system to be working properly. Now, it's not always the case that there is anything at all out there in the world that would explain what my perceptual system is telling me. Could be telling me all sorts of inconsistent stuff. But in this case, if there had been little pink elephants floating out there in the world, my perceptual system would have been telling me the truth.

[00:40:33]Okay? So, I can now say exactly the same thing to As I said before, without using the words subjective experience, I can say to you, I dropped some acid and my perceptual system telling me fibs, but what it's telling me would be correct if there were little pink elephants out there floating in the world. Okay, so these little pink elephants aren't funny things in a theater made of spooky stuff called qualia. They're hypothetical things in the real world. But the pink and the elephant and the floating are all normal pink and elephant and floating. It's just they're not actually there. They're hypothetical.

[00:41:11]So what's funny about a subjective experience is it's something hypothetical, not real. Not that it's made of qualetta.

[00:41:19]At least that's the view I'm trying to persuade you to believe in. Um and I got ahead of my slides here because I got so carried away with this.

[00:41:29]Okay. So, it's just an indirect way of me telling you about what's going on in my brain. Obviously, if I told you neuron 52 is firing, that wouldn't do you any good because you you knew it would be neuron 57. And anyway, I don't know neuron 52's firing. That's a lively way for me to tell you what's going on in my brain. The only way I can tell you what's going on in my brain is by talking about either the normal things that would have caused it, these little hypothetical little pink elephants, or I could tell you about what's going on in my brain by telling about the normal things it would cause.

[00:42:04]So, I can say, um, if you ask me how I'm feeling, I can say, I feel like punching Gary on the nose. Um, so feelings are all about describing what's going on in your brain by talking about hypothetical actions and percepts or subjective experiences talking about hypothetical inputs. Okay, I'm getting ahead of myself.

[00:42:27]So I'm going to now show you a multimodal chatbot. Let me just show you on a slide a multimodal chatbot having a subjective experience.

[00:42:37]So I take this multimodal chatbot and it's got a camera, it's got a robot arm and um it can see.

[00:42:48]So I train it up and I put an object in front of it and I say point at the object points at the object. No problem.

[00:42:59]I then put a prism in front of its lens and when it's not looking and I put an object in front of it and say point at the object and it points over there and I say no that's not where the object is the object is actually straight in front of you but I put a prism in front of your lens and the chatbot says oh I see the prism bent the light rays so the object's actually there but I had the subjective experience erience it was there. Now if it uses the word subjective experience in that way it's using them exactly what like we use them. So a chatbot that said that would have had the subjective experience that it was there. The prism messed with its perceptual system. It wanted to tell you what was going on in its perceptual system. And the way it could tell you is by telling you what would have had to been in the world had we not messed with its perceptual system. Um, that's what it's telling the chatbot. Um, okay. So, my claim is multimodal chat bots already have subjective experiences. Now, as you can imagine, subjective experience is kind of the thin end of a wedge. I chose to talk about subjective experience because it's sort of cleaner than talking about sentience or consciousness. Um, many people are very confident that chatbots don't have aren't sentient. But if you ask them, "What do you mean by sentient?" They say, "I don't know." But they're very I'm I know they ain't got it, but I don't know what it is. Um that seems to me not a very sensible position to hold.

[00:44:35]Um the other thing I might talk about is consciousness. Consciousness is more complicated because it typically involves um you having a model of yourself in a way that subjective experience doesn't involve so much. So it's easier to talk about subjective experience. But my hope is that if I've unsettled your very strong belief that there's this inner theater and I have experiences in their mind and they're in this inner theater, there things in this inner theater. Once I've unsettled that belief and once you once you begin to get over that, um you'll be able to see that it's perfectly reasonable to think these things are conscious. Um I was once I'll end with an anecdote.

[00:45:18]I was once um visiting Microsoft in Seattle and I couldn't sit down so I got a train there and I got a taxi from the train station to Redmond where their lab was and he was going across some he's on a freeway going across some big bridge and the taxi driver was a Somali immigrant who'd recently immigrated from Somalia and to make conversation he said um what's your religion? So I said, "Well, I don't actually think there's a God."

[00:45:48]And the taxi driver, he was going about sort of 60 miles an hour. The taxi driver turned round and he stared at me in total astonishment like he he never thought he'd ever meet someone who didn't understand that God runs things. Um he was just utterly utterly amazed. Um, now he probably only turned around for about three seconds because I'm still here, but it seemed like it seemed like a very long time. Um, that's what many of you will be feeling hopefully. Um, I want to you to realize you're as wrong as that taxi driver was.

[00:46:30]Okay.

[00:46:33]Um, actually that was just a joke and you laughed, so we're done.