Beyond LLMs: Why the AI Industry is Now Backing Yann LeCun’s 15-Year Bet.

[00:00:00]I'm particularly interested in uh how models are going to uh evolve to allow humans to interact with them better. I think we're all, you know, very comfortable chatting with uh models with text, but uh as humans, we have many ways of communicating. And I think we're going to find AI architectures that are uh much more fluent in in being human and allow us to communicate with computers in a in a much different way than we do today. And that that will require new architectures. So I tell people in AI you should stop doing generative generative AI or generative models and of course it doesn't make me very popular.

[00:00:36]>> Well young lun should feel popular again after this endorsement from Peter Deantis senior for foundational AI models custom silicon and quantum computing at Amazon. But the Amazon executive is not the only one to share Lun's vision. More big tech companies are now working on world models and we will see the details shortly.

[00:00:58]For the last 15 years, Young Leon has been working on models towards AI super intelligence but following a completely different path that does not revolve around LLMs but on videos as a representation of the world. Leon has been very outspoken about this vision on LLMs and at least initially his stance made him something of an outcast in a field dominated by text. So let's hear why the industry is so hooked up to LLMs and why according to Lun they are not the future of AI. So those things are useful. They're powerful. Um they are superhuman in some domains, but they have limits. And the idea that somehow those limits are are going to be expanded to kind of cover all the things that humans do is just false. There's a term to designate uh people who believe that we're going to, you know, scale up LM and get to super intelligence. They call LLM pill, right? they took a pill and now they're hypnotized. Uh and to some extent a lot of the culture in Silicon Valley or in the AI industry is LLM pill. And the the funny thing is that because all the companies are pushing each other's engineers and they're all working on the same thing, they can't really afford to sort of deviate from the mainstream because they run the risk of falling behind and so that creates this monoculture. Um but I think it's a bit of a delusion and uh you know I think there is requirement for for you know new paradigms to kind of go beyond the limitations of current systems and there's many ways to get to this new paradigm and perhaps building up LLM is one way but I think it's kind of a slow way. So I tell people in AI you should stop doing generative generative AI or generative models and of course it doesn't make me very popular. I mean there there's two areas where LLMs are geniuses.

[00:02:58]>> Yeah.

[00:02:59]>> It's mathematics and code >> and those are two domains where the the mere manipulation of symbols is actually the substrate of reasoning. It's not it's not a coincidence. LM are good at manipulating symbols.

[00:03:13]>> So they they work well for math and code. Two things. First thing is if you want to build an agentic system, everybody has been talking about agentic system, right? Uh so an agentic system is a system that produces actions either in the real world, the physical world if it's a robot or in the digital world. In my opinion, you cannot build a reliable agentic system without this system having the ability to anticipate the outcome and the consequences of its own actions.

[00:03:46]We have this most of us maybe some politicians don't but but we are certainly capable of anticipating the the outcome resulting from our actions and that's what allows us to plan a sequence of actions to accomplish a task and so that's what a world model is given the state of the world at time t given an action that you imagine taking can you predict the state of the world at time t plus one where one means you know 10 millisecond a second a minute an hour or 10 years.

[00:04:15]>> Yeah.

[00:04:16]>> Um that's what a world model is. And so that's the first thing. Now people say, well, I can do this with an LM. I can give a text description of the world and then a text description of an action and then a text description of the next state of the world and then train a system to just predict the next description from the first two. Right?

[00:04:37]LM in principle could do this. It can do this as long as you can describe the world in terms of sequences of discrete symbols.

[00:04:45]Um, and the reason for this is predicting discrete symbols is difficult. You can never predict which word will follow a sequence of words.

[00:04:57]But you can predict a probability distribution over all the possible words in your dictionary. And then by auto reggressive prediction, you imply a distribution over sequences. Right?

[00:05:06]That's how work. You simply cannot do this with real data. So if you have data from sensors that are continuous, highdimensional and noisy, you simply cannot predict what's going to happen. The example I use all the time, which is particularly appropriate in this room, is if I take a video of this room and I turn the camera slowly. I stop the the the video here and I ask the system to predict what's going to happen next in the video. It can predict there's going to be more people sitting in the chair. It can't predict what the size of the room is.

[00:05:37]And there's no way you can predict what all of you look like. Absolutely no way.

[00:05:41]So if you train a system a generative architecture like an LLM to predict in every detail the at the pixel level uh you know the continuation of that video it will fail. It's just not possible.

[00:05:55]And in fact it's a mathematically intractable problem because we don't know how to represent distributions over an infinite number of plausible outcomes.

[00:06:04]Yan Leon however should find some satisfaction in knowing that more and more big tech companies are now sharing his vision. But before looking at who else is taking the same approach, let's listen to Luna explaining the advantages of his world model and the latest updates delivered at Vivate 2026.

[00:06:23]And so the idea that we're building ammy ami labs on is the idea that the system should find an abstract representation of the observations and make predictions in that abstract representation space without attempting to reconstruct all the details which really are not predictable. It's very fundamental.

[00:06:45]>> How hard is that to do? How how long a project is that before you have something you know that that's you know you say you've been working on it even at meta but yeah how close are you to actually saying okay here it is you know folks what's to do with >> well I've been working on the idea of using self-s supervised learning by video prediction for about 15 years mostly failing for the first 10 because I was using generative models trying to predict at the pixel level uh and then five years ago we came up with a number of ideas and also that was prompted by some results in uh image recognition where the the best architectures that are trained by self self-supervised learning to learn image representations do not attempt to reconstruct. They're called joint embedding architectures. Um and so it was pretty clear to me there was a clear message that architectures that are trained to reconstruct don't work and the ones that are trained to just run a good representation that has some interesting property work better.

[00:07:42]They say well you should apply this to video too. And so that's what happened about 5 years ago. There's a difficulty a technical difficulty there which is to prevent the system from collapsing. Uh so when when you tell the system okay here is a piece of video find a good encoding for it and then here is the following piece of video encode it as well and now predict the representation of that second chunk of video from the first one. If you train all the thing together without being careful, you just train it to minimize prediction error system will just ignore the inputs produce constant representations and now the prediction problem is trivial but the thing doesn't do anything for you right uh so how do you prevent this collapse and about you know four or five years ago we discovered a series of technique basically kind of a new way of doing this uh one of them is called sigg another one is called vcg they're basically based on the idea of maximizing some estimation of information quality and those work really well. So because they work well, we think we can scale them up. We can apply them to a lot of different problems. This architecture by the way is called JEPA, joint embedding [clears throat] predictive architecture and it's not generative. So so with this kind of technique we can we can train a system to learn an abstract representation of observations that are complicated.

[00:09:00]train it to produce a one model which is again state of the world an abstract representation of an observation and then from the state of the world and an action predict the next state of the world and then uh use this to plan uh a by optimization find a sequence of actions that will accomplish a task and we're working on systems of this type that are hierarchical I'm not going to go into the details but um but we have good hopes that this will really be a a complete change of uh the blueprints of intelligent systems that will perhaps take us to you know considerably more intelligent systems that can you know anticipate the outcome of of their own actions can plan can reason and uh perhaps all the way to human intelligence but that's going to take a while it's not going to happen next year we're not going to have a country of geniuses in a data center next year >> now the scenario of a world with super intelligent machines and where everybody's is a genius rises some concerns.

[00:10:02]Luna has something to say to those that express fears about a future dominated by AI and he seems to address specifically to Dario ammo day of entropics. Here is Lun's response to those that see AI as a threat to our civilization. A lot of people are particularly people who are close to the US government which for all of its characteristics is actually somewhat in favor of open source AI and and they they see that American industry is not doing open source anymore and that the open source uh scene is uh basically entirely occupied at least at the frontier by Chinese companies and what you see in the utilization of um of of [clears throat] you know AI platforms um there's a huge increase in the use of Chinese models which are pretty good and they're cheap and you know cheap to run >> right >> they're free if they're open so so that creates kind of a conflict >> and in the middle of this you have you know companies like anthropic and a few a few others that are basically lobbying to essentially make openource illegal, right? Because they think AI is intrinsically dangerous. Uh this is like one of the huge debates where very strong opinion as you probably know. I think we are in a debate similar to what was probably happening in the 15th century when the printing press was invented >> where the Catholic Church didn't like it. The Ottoman Empire hated it. Uh but you know in in Europe at least they couldn't stop it. Right? So the people weren't printing books and people were able to like read the Bibles by themselves. That could that caused the Protestant movement in Europe uh 200 years of religious conflicts. Okay. But also people like learn to read because now they had something to read and and ideas about you know science and philosophy and democracy and things like that were disseminated. So that brought about the enlightenment and the American and French revolutions uh the the basically destruction of the feudal system in Europe. Right? So I mean this those things have huge effects and it's a cultural uh medium that disseminates knowledge. Now what is AI today? AI today LLM are a way to disseminate knowledge. They're not yet a way to generate new knowledge. They're really a way of kind of building repositories of all human knowledge and then giving access to people in a sort of interactive way.

[00:12:46]So it's just another step in, you know, making people smarter by disseminating knowledge. So if you're going to block this because you think it's dangerous, you are being a medieval obscurantist.

[00:12:59]Uh and I think that's insane, >> right? Um, so, so they want to preserve the dogma.

[00:13:07]Maybe they want to control what people can do with with their tools. You know, if I, uh, if I buy a, if I buy you a pen, you know, I don't want the company that builds the pen to tell me what I can write with it. And I'm not a writer.

[00:13:17]You are. Um, so, uh, you know, you there's a limit to to to what what a company should be able to to do with uh or authorize others to do with their tools. Um but but I think the you know the the dissemination of of of knowledge is and culture I think is intrinsically good. There are dangers attached to it.

[00:13:40]You mitigate them. Um but but otherwise it's just uh it's just control. And I think it comes also from I mean there's a discourse where okay you know it's too dangerous to give AI to everyone. Um but we can do it like we can use it because you know we're smarter than everyone right? I mean there's there's a big arrogance and superiority complex in the idea that only a few are capable of controlling AI and the unwashed masses should not have access to it. Uh they are dangerous but you you just you know you just correct for them. I it's just like you know when email first appeared there were scams and there were all kinds of stuff and you know and spam and everything and you know we find a solution to that. And here is a summary of the other big tech companies that are developing wall models as an alternative to LLMs.

[00:14:31]We start with Google the mind who's working on Gene a model that is also trained entirely on raw video to generate interactive 3D environments.

[00:14:41]Then there is the world labs founded by FA one of the pioneers of computer vision and who is now working on the concept of spatial intelligence.

[00:14:52]And then there is Nvidia that is using video for its own cosmos.