How machines become minds | Geoffrey Hinton & Joel Hellermark

Added: 2026-06-02

[00:00:05]Couldn't be more excited to be back with Geoff.

[00:00:10]Last time we spoke, I promised you you would get the Nobel Prize last year.

[00:00:14]You thought it was a very very low probability.

[00:00:17]Did you tell the Nobel Committee that as well?

[00:00:20]Actually when they called up I didn't believe them.

[00:00:24]I thought it was a prank.

[00:00:26]Nobody had mentioned that they'd nominated me.

[00:00:29]There isn't a Nobel Prize for computer science and actually I went round for several days after that doing a little Bayesian calculation.

[00:00:36]That is: "what's the probability that a theoretical psychologist hiding in computer science will get the Nobel Prize in physics?"

[00:00:44]Well, maybe one in two million.

[00:00:46]Now what's the probability if this is my dream that I get the Nobel Prize in physics? Well maybe one in two.

[00:00:52]So that means it's a million times more likely it's a dream than reality.

[00:00:58]And I remember when I was a kid I used to dream about flying.

[00:01:01]Most kids do, I think.

[00:01:04]And I would occasionally have dreams where I remembered that last time I could fly it turned out to be a dream.

[00:01:08]But this time it wasn't.

[00:01:10]And so I was actually worried for several days after I got it that it might be a dream. And I contented myself with one thing.

[00:01:17]If it was a dream I would wake up and that nightmare about Trump being president wouldn't be true.

[00:01:29]Unfortunately you won the Nobel Prize and Trump was the president.

[00:01:34]I'd give it up for that.

[00:01:40]Maybe he'd make the trade actually.

[00:01:42]We should take this idea seriously.

[00:01:44]Well he obviously deserves one.

[00:01:50]But you've also been spanning the ideas from cognitive science to computer science. And one thing I find is quite fascinating is at some point sort of human intelligence reached diminishing returns.

[00:02:03]We didn't get that much more intelligent than that.

[00:02:06]We could have just hill climbed for you know a couple more million years and become increasingly intelligent.

[00:02:11]Do you think we're sort of intelligent enough and it doesn't make sense to make humans more intelligent?

[00:02:18]My guess is that we're—we've got to a point in intelligence where we're going to create something else that can get intelligent much faster.

[00:02:27]So if you look at evolution, you start off with things just doing evolution.

[00:02:31]Then they sort of get development. And development has a much—has a 20-minute cycle time as opposed to evolution which has a lifetime cycle time.

[00:02:38]And then after you've got development you then get learning. And development has a 20-minute cycle time whereas learning has a kind of one second cycle time.

[00:02:48]So what happens is we keep developing faster and faster in the loops, and now what we've developed is AIs, which I think are the faster in a loop and they're gonna get much smarter.

[00:02:59]Where do you think this will take you?

[00:03:02]Do you think we will see sort of 300 IQ models and they would be vastly more useful or do you think it's more parallelizing and being able to think a million thoughts in parallel and pick the best ones?

[00:03:14]Do you think the intelligence in itself or the parallelization is the most useful thing?

[00:03:18]Both I think.

[00:03:20]So if you look at Go or chess they've got the 300 IQ there already right?

[00:03:26]No person will ever beat them again, at least maybe one game, but I think it's going to get much more intelligent than us that's my guess.

[00:03:35]So I think today it was announced that, by the mathematicians, that it proved one of Erdős' theorems and it wasn't something that it found somewhere in the ancient literature. It was something actually figured out in a quite clever way using a different branch of mathematics.

[00:03:51]So I think that's the sign that within mathematics at least it may well get much smarter than people because it's a closed system.

[00:04:00]So just as it did with chess and Go it can just make conjectures, see if it can prove them, and just keep playing around like that and by playing like that it'll get smarter and smarter. And I think it may well outstrip mathematicians fairly soon, like in the next 10 years.

[00:04:15]And I'm surprised we haven't seen more examples like this yet.

[00:04:19]It seems intuitive to me that if you had all of the world's knowledge in your head at the same time through analogy you should be able to discover quite a lot of new knowledge. What do you think is shifting and do you think this is sort of a trend we should expect now with a lot of new knowledge being generated?

[00:04:38]So I think if you just use language to train, it's amazing you can deal with spatial things at all.

[00:04:43]I mean, it's a weird fact.

[00:04:44]If you just train on language, you do understand quite a lot about space just from language. But obviously you understand more about space if you can pick up objects and move them around and feel inertias and things.

[00:04:57]So I think that's one deficit that the models have had until recently.

[00:05:03]People have that.

[00:05:05]I think there's still a lot about people we don't understand and there's probably still a lot of improvements we can make in the AIs.

[00:05:11]But I'm a materialist all the way through, so I don't think there's anything about people that we won't be able to get in AI's.

[00:05:23]I would like to talk—to say I listened to the last talk, and I quite like people.

[00:05:32]And so, as we make these models increasingly better, what do you think now is the delta between Albert Einstein and these models?

[00:05:43]Is there something fundamental about these models that doesn't allow them to just do these thought experiments and generalize fantastically well from those?

[00:05:53]Or should we start to expect to see some of these quite clever thought experiments that then extract into a range of domains?

[00:06:00]I think in the longer run we will see that.

[00:06:03]Maybe not in the next few years, but if you think about the next 20 years, I think we'll be seeing things like that.

[00:06:08]I think if you look at human history, recent human history, there was the Copernican Revolution and it took people a long time to accept that the earth wasn't the center of the universe.

[00:06:21]Particularly religions didn't like that at all.

[00:06:27]Then there was the Darwinian Revolution.

[00:06:28]It took people a long time to accept that we were animals.

[00:06:32]We're very special animals because we've got language, and we got very big brains, but we are animals.

[00:06:38]I think we've got a new revolution coming, when we're not the only beings around. I think we're creating beings. And it's going to take people a very long time to become happy to say we're creating beings.

[00:06:54]Right now, people are reacting just like they did with Copernicus and with Darwin and saying "No, no, no, no, no.

[00:06:59]I don't want—I mean I don't believe that, that's crazy."

[00:07:03]There's something really special about people.

[00:07:06]I mean I think people are very special to other people, but I don't think there's anything about us that the AIs won't get in the end.

[00:07:14]Benjamín mentioned some of the work on AlphaGo, where eventually they went from AlphaGo to AlphaZero and they sort of skipped the pre-training step and went all in on purist reinforcement learning.

[00:07:28]Do you think that makes sense to do more generally, as we shift the training paradigms of these models to be more purist reinforcement learning, or do you think inherently it's very useful to have this pre-training?

[00:07:41]The pre-training makes things much more efficient.

[00:07:44]I think in closed worlds like mathematics or a game, that approach will work.

[00:07:53]And I think if you look at language models now, the language models now are like the early Go models were.

[00:07:59]And the early Go models—you've got a neural net to try and model the intuition of a Go player by getting it to try and copy the moves a Go player would make.

[00:08:08]So the database of all the moves experts made, you tried to get your neural net to pick the right move. And that's going to be limited by the data you've got from the experts and by the skill of the experts.

[00:08:20]As soon as they stopped doing that— they use that to initialize it, but as soon as they stopped doing that and started doing Monte Carlo rollout so they get their own training data, the things then took off.

[00:08:31]The equivalent for language models is going to be, you have some beliefs, you do a bit of reasoning with those beliefs, you arrive at a new thing you should believe, but you don't believe it. And now you've got an inconsistency.

[00:08:45]And so either the way you did the reasoning was wrong, and reinforcement learning can fix that, or you should change the premises, or you should change the conclusion.

[00:08:54]And so now you've got a system that can improve with no extra data.

[00:08:58]It can generate these internal inconsistencies, and I think that means these language models can get hugely smarter without a lot more data, and Demis thinks the same thing.

[00:09:09]So I assume Gemini is already doing stuff like that.

[00:09:12]And as these models become better, do you think that there will be a compounding advantage? We're already seeing this to some extent.

[00:09:20]Where the models get better, they use the models to improve the models, and it also has these sort of side effects, with the best talents joining the labs and so on.

[00:09:31]Do you think we'll see that sort of takeoff where being six months behind is not that valuable and there will be one lab that compounds this all the way, or do you think it will be equally distributed across four or five labs?

[00:09:46]So, about a year ago, Eric Schmidt was talking about this a lot, about how being a year behind was no good because if it takes off, you'll just be left in the dust.

[00:09:57]I just don't know.

[00:09:58]I really sort of feel I don't know.

[00:10:02]That's a possibility.

[00:10:04]I hope it doesn't happen like that.

[00:10:06]My gut feeling is it probably won't happen like that.

[00:10:10]We won't get one thing that just gets much better than all the others very fast.

[00:10:13]They'll all get better. They'll all start using the AI to come up with new ideas for the architecture and stuff.

[00:10:23]So you'll get this recursive loop.

[00:10:25]I really just don't know the answer.

[00:10:30]If you were a lab leader, would you put more of your eggs in the same basket and just double down on what's currently working?

[00:10:39]Or would you place a lot of parallel bets on the new model architectures?

[00:10:44]This is a counterfactual.

[00:10:45]And along with if you were a lab leader, we have to say what my motivations are. If I've got stock options, if I want to get to a trillion dollars quickly, then I would double down and just build a huge computer and get on with it.

[00:11:02]If I was interested in the future of humanity, I think I might try lots and lots of bets in the hope that we could develop better beings.

[00:11:13]You see, I think what's happening at present is everybody is trying to make profits so they can go public.

[00:11:21]And they're not thinking about the other aspects of beings.

[00:11:26]They're thinking about how to make these things more intelligent.

[00:11:28]Everybody's going for more intelligence.

[00:11:31]But if you think about a being, there's a lot more to a being than intelligence.

[00:11:35]And we should be very concerned, we're making these beings, and we should be very concerned to make them beings that care about us.

[00:11:43]And we can still do that.

[00:11:44]But nobody's putting much effort into that.

[00:11:46]What's happening is—if you ask "Where did we come from?"

[00:11:50]We came from a kind of invisible hand of evolution.

[00:11:53]There was competition between warring bands of chimpanzees, well our common ancestors were chimpanzees.

[00:11:59]And that led to all sorts of things like loyalty to your own tribe and a willingness to be extremely nasty to the other tribe.

[00:12:08]Caring for your children, intense loyalty to a strong leader, stuff like that.

[00:12:13]A strong willingness to collaborate with people within your own group.

[00:12:18]That was all from the invisible hand of just competition.

[00:12:22]Now what's happening—but it didn't make us nice.

[00:12:26]It made us nice to people in our own tribe and nasty to people in other tribes.

[00:12:32]What we want, if we're creating a new kind of being, we don't want them like that.

[00:12:38]We don't want them nice to them and nasty to us.

[00:12:41]We want to create a kind of being that cares primarily about people, much more than it does about itself.

[00:12:49]And we're not going to get that by kind of the invisible hand.

[00:12:52]Because what we've got now is we've got this competitive race between companies to make the smartest possible AI that can do the most things.

[00:13:00]That's going to lead to things that aren't nice beings towards us, I think.

[00:13:08]What we want to do is be designing them so that they're going to be nice towards us. And I think we should be putting a lot of effort into that.

[00:13:15]So, I have a theory here.

[00:13:19]The fact that you are saying that the AI will turn evil is increasing the probability of the AI becoming evil.

[00:13:28]The AI is out there reading the internet and it's saying, "My creator Geoff Hinton says I will be evil."

[00:13:34]It's like listening to your parents all day say that you're going to be the next Hitler.

[00:13:40]My parents didn't say that.

[00:13:45]Should we explicitly remove this from the training data?

[00:13:48]Because now it's reading the training data.

[00:13:50]It's seeing all of its evil behavior be continuously discussed on the web.

[00:13:54]But it might also make them think we ought to focus on how can we be nice to people.

[00:14:01]I mean, I think we should be putting a lot of effort into that.

[00:14:04]What would you change about the existing model architectures?

[00:14:10]If you were placing one last bet on a new model architecture, what would it be?

[00:14:18]Yeah, I'm too old to have a novel idea there.

[00:14:21]But I think what they're missing at present is fast weights.

[00:14:26]So you want to do efficient matrix multiplies.

[00:14:30]You're going to take a stack of vectors that say—neural activities on different training cases. And you're going to multiply them by a matrix of weights.

[00:14:38]And it has to be the same matrix for every vector.

[00:14:42]And that means you can't have weights that are changing as you go through a sequence of sentences.

[00:14:51]Because your weights need to be the same for all the sentences because you're running them all in parallel.

[00:14:57]We clearly have fast weights.

[00:14:59]Most of our synapses adapt quickly and decay quickly.

[00:15:04]My friend Terry Sinofsky, who's my sole source for neuroscience information, says only about 10% of the synapses are for long-term learning.

[00:15:16]Most of them are for short-term things.

[00:15:18]And that's what we don't have in these models.

[00:15:22]Now, what we do have in the models is the ability to have copies that go a long way back of the neural activities at many different times.

[00:15:31]And our brain doesn't have that.

[00:15:33]So how does our brain manage to compete with transformers if we don't have those copies? Well, the way it does it is by having fast changes to the weights so that in that weight matrix, you've stored lots of information about recent history efficiently and you can access it efficiently to get what's similar to things you're seeing now. I think there will be progress in doing that.

[00:15:57]But I think that progress may only come when we have computers that are using things like voltages and conductances for neural activities and weights.

[00:16:06]As long as we're using digital computers with matrix multipliers, there's going to be too much a win from not changing the weights as you go.

[00:16:15]So that would be my last bet.

[00:16:18]And what do you think in the training data that we actually put forward towards these models? Do you think it matters and we should make it much more curated?

[00:16:31]Or do you think we should just scale it ubiquitously?

[00:16:35]Would you teach your child to read on the diaries of serial killers?

[00:16:40]Probably not.

[00:16:41]There you go. There's your answer.

[00:16:42]But isn't this a fundamental issue in how we scale this?

[00:16:49]Yes. I mean, look, we know that these large models are much more like people than they are like standard computer software.

[00:16:55]They're not lines of code where each line of code does something that somebody thought it was meant to do and maybe doesn't.

[00:17:00]They're a whole bunch of weights that is learned from data.

[00:17:04]And if you ask, "What control do we have over that?"

[00:17:06]Well, there's two main controls we have.

[00:17:09]One is we can reinforce it.

[00:17:11]We can tell it don't do that, or do do that.

[00:17:15]You can do that with people too.

[00:17:17]But there's something that's much more effective than that with people, which is modeling good behavior.

[00:17:21]If you want to raise a decent child, you should just look to see what happened when Trump was raised and do the opposite.

[00:17:30]So we should curate that data a whole lot so you model decent behavior to it.

[00:17:37]Maybe this is a company we could build.

[00:17:41]Okay, I have a final question for you.

[00:17:44]What is the difference between Oppenheimer and Geoff Hinton?

[00:17:48]Oh, before we came on stage, I told him a joke.

[00:17:54]Just after my Nobel Prize was announced, I was giving a talk to a bank for a very reasonable fee because I hadn't had the Nobel Prize then.

[00:18:03]And I had at the beginning of the talk, because they talked about the Nobel Prize, of course. At the beginning of the talk, I said there's no comparison between me and Oppenheimer. Oppenheimer was a brilliant physicist and also a brilliant organizer. I'm a terrible organizer and I'm just quite a good scientist who happened to pick the right problem and stuck with it.

[00:18:24]My special ability is stick with it.

[00:18:27]It's the ability to ignore other people.

[00:18:31]So anyway, at the end of the talk, there were questions.

[00:18:34]And the last question was, later in his life, Oppenheimer came to regret what he'd done.

[00:18:40]How would you compare yourself with Oppenheimer?

[00:18:43]And I couldn't resist it.

[00:18:45]So I said, Oppenheimer never got the Nobel Prize in physics.

[00:18:56]I think that's a good note to end on.

[00:18:58]Thank you so much for joining us, Geoff.

[00:19:00]It's always a pleasure.

[00:19:01]Thank you.