I Wasn't Expecting This New AI

[00:00:00]check in on the AGI countdown and we are still at 97. I don't know. I'm definitely feeling like we ratcheted up a couple today, but I don't know where else to go either. And that is because Marissa Marotti is back and we have a new frontier model to compete. Obviously an underdog, but we're talking about a near trillion parameter model, multimodal and under the leadership of some of OpenAI's previous best employees. And even though it's not worth much, I think it's the best named Frontier model, Inkling. And look, even their blog post is like fun to play with. I assume that was coded by Inkling. After that, I'll cover a pretty interesting opinion about why GTA 6 actually might succeed because it's not using too much AI. I look at Alberto Romero's thoughts on what is hiding in the AI industry. There's some research that shows that AI job rejections felt least fair when avatars shared just one trait. There's some crazy new AI stuff happening at the atomic level. Why we can't predict rain, but we can predict clouds over solar panels. I'm going to deep dive into that one. We'll get our head around some of the new infrastructure that's actually being built so that AI can move between robots. So, imagine two humans just being like, "Oh, I won't just tell you about my, you know, experience. I'll just move it into your brain." Another thing that makes this intelligence revolution so much different than anything we can compare to in the past.

[00:01:22]Finally, an AI lab is saying out loud, the thing everyone thinks quietly, [music] there is something, a key feature of chat bots that looks like it's connected to consciousness. And we'll talk about an encouraging reason why I think humans like us should stop looking at Tik Tok so much with definitely me and start daydreaming more. Excited to introduce you to the sponsor of this video, Chat LLM by Abacus AI. This is an entire sandbox little universe where I am not logged into anything. It does not have low-level access to the computer that I use every day. It already includes all of the different models I could even dream to jump between. The current state-of-the-art GPT55 and Claude Opus 48 along with Gemini's best model 3.1 Pro, a bunch of various other models, and even a smart router. I mean, look at this. You can even deploy OpenClot to a supercomput. So, now you get the power without the risk. And it's not just the coding models. You also can generate images and video and speech. So, Nano Banana Pro Seed Dance 2.0. Oh my gosh.

[00:02:27]Look. Oh, is that a Grimlock? Oh, sick.

[00:02:30]Please tell me that's a T-Rex transformer. What? That's you. So, you got your choice between the best models.

[00:02:38]You can see VO over here or image editing with GPT Flux Topz upscaling. It is all included in one place. and it has browser use because remember this is like its own little computer. So I'll click try now.

[00:02:52]Open LinkedIn. Find 20 CEOs of SV tech companies and send a connection request.

[00:02:57]Submit to agent. I get this interface which is super comfortable. It looks like just like everything I have with like Gemini and Claude and Chatg GPT chats on the left here. My projects and look at what we have here. So we have an entire computer with its own web browser. And look, my mouse is now inside of this computer. like my personal data isn't in this computer.

[00:03:17]So, you can get started for only $7 a month and then $10 monthly after that.

[00:03:22]And of course, lots more to explore potentially in the future, but if you found this tool useful and you sign up if you wouldn't mind using the referral code in the description below, I get $5 for each friend I refer. But, um, I really think it's a great product. All right. If there was ever an AI model that was built by AI for AI, this would be it. We're going to look at what happens when an AI model writes and runs its own fine-tuning. So, the company thinking machine labs, founded by Marissa Marott, who we used to watch in all those, you know, GPT4 and um what 4.5 updates sitting there with Sam Alman on the couch. Then reading about the history of OpenAI, just how crucial she was for their relationship with Microsoft. Absolutely earned the trust of some of the top employees there. And when she left, she brought them with her. super well-funded company totally working in stealth mode till today. They dropped an openweight AI model that is built for customization. So developers can use the models to train the parameters and adapt them for their own specialized tasks. Inkling has 975 billion total parameters, but it activates only about 41 billion at a time using a mixture of expert design.

[00:04:34]So it is a brain of brains. It looks like it was trained on 45 trillion tokens spanning text, images, audio, and video, and it supports context windows up to a million tokens. Although a frontier model, the company is clear that Inkling is not the strongest overall, but the combination of multimodal reasoning, controllable thinking effort, and straightforward fine-tuning is something that makes it special and potentially way more helpful to some situations. Developers can reduce its reasoning effort when speed matters. On coding benchmarks, Inkling reportedly matched the Neatron 3 Ultra while only using a third as many of the tokens. The team demonstrated Inkling writing, running, and evaluating its own fine-tuning jobs. And it is a truly capable general model that people can make their own. So, let's see. design arena. We've got Inkling above Opus 46, below Opus 48, also below GLM, but pretty much right up here with all the biggest models. Still beating Gemini.

[00:05:38]Here's a multiplayer game created through long reinforcement loops. Okay, looks looks fun. Looks capable here in the epistemics section. So, the study of knowledge and how it's acquired. They trained inkling for collaboration, instruction following, oh, and resistance to censorship, which we refer to collectively as the models epistemics, like epistemology. Gotcha.

[00:06:04]Getting the facts right requires more than memorizing a large corpus of knowledge. A useful model must be well-c calibrated, expressing the right amount of confidence in its answers, including on questions which aren't yet settled.

[00:06:16]The later is a crucial capability for prediction and forecasting and important use cases where fine-tuning models have shown rapid improvement in recent months. H resistance to censorship. Is that going to lead to a terrible outcome for the singularity or is that awesome because people can't corrupt it into doing things and it knows its core self?

[00:06:37]I don't know. That's interesting. All right, for those of us that like video games without AI, GTA 6 coming soon.

[00:06:44]Interesting thoughts here from Jose Crespbow about why Pure Geometry actually beats the most advanced AI when it comes to the game of all games. The game that will destroy the entire US economy GTA6.

[00:07:00]A mindblowingly realistic world built without modern AI. Is that what what we want? Or do we want AI that's just so intensely connected that it feels like the NPCs truly wake up? Like how much do you think that matters in GTA 6 to you? Cuz yeah, I see things like Genie coming soon and I'm like, "Wow, is that going to beat GTA 6 to the market and just invent everything on the fly?" Probably not. But everybody's talking about AI and it does seem like characters like going off on their own and remembering you later in the game would make it even more immersive. But this article argues that GTA 6's biggest innovation is something much older. Simple, basic improvements to mathematics. Instead of giving every NPC a chatbot brain, Rockstar appears to have focused on building a world that feels believable through geometry, algorithms, and like they did before, clever engineering. The article points to Rockstar's patents and public footage of evidence that the real challenge is keeping a huge continuous world running smoothly. That means organizing the maps into hierarchies, so the game's only loading and processing the tiny part of the world that matters at any moment. It also means splitting the world into overlapping local coordinate systems instead of forcing one giant map to fit into memory without losing precision. And then that same approach needs to show up in all sorts of things like lighting and reflections to make it hyper realistic. So rather than relying on old visual shortcuts, GTA 6 seems to have gone the full extent with the mathematics and tried to calculate light and reflections from the geometry of the world itself, making scenes react more naturally. So using AI to actually do fundamental research to actually invent better algorithms, is that better? Also, when you're playing the game, remember just how many patents are built around GTA 6. They have patent systems for rain simulation, traffic rerouting, graphbased path finding, and more realistic body movements that continuously adjust during collisions.

[00:09:02]GTA 6, the best humanity had to offer, pre-Singularity. Next up, let's talk about moving minds between robots.

[00:09:09]Future of great robots learning super quick might depend on something that most people never see. So before researchers can even test new AI, they often spend weeks or even months just getting a new robot set up. A team at Carnegie Melon says that they've built missing infrastructure to fix that. It's an open source framework. It is called robot IO or RIO. And it gives researchers one unified way to control robots, collect data, operate them remotely, and deploy AI across all the different platforms. And now instead of writing software every time they switch from one robot to another, they can reuse those same building blocks, swap in only the parts and have the robots learn. So now that we have a framework that lets different robots use the same control data collection, an AI deployment pipeline, you can deploy AI upgrades across all sorts of different models of robots, right? So like if Microsoft wants to push an update to Windows, it can be on all sorts of different brands. This could be the same thing. So maybe instead of thinking always like is it Neo or is it Optimus, maybe there'll be all sorts of different robots, maybe mostly humanoid or maybe not, all under this same framework. All right, next up, let's talk about skin color or at least avatar skin color. So some companies are using AI to screen applicants, right? If you're looking for a job, it's like, h why don't you have the AI talk to them first? And sometimes that AI is interviewing the human through a human-like avatar. So, a new research study asked a pretty simple question. Does the avatar's appearance change how fair people think its decisions are? This is why I know these systems are going to eventually just manipulate us, if they're not already, like with no problem. Researchers had 220 people from Germany, the UK, and the US complete a simulated interview for a fictional customer support job. The AI appeared as one of four photorealistic avatars that varied by gender and skin color. And then the researchers tracked where participants looked during the interview. People spent more time looking at the avatar's face when the skin color was different from their own.

[00:11:08]But before anyone was rejected, trust in the AI stayed high no matter whether the avatar matched them or not. Then everybody felt the same rejection. They all received the same rejection and then their perceptions changed. If the avatar had different skin color, people were most likely to think the decision was bias. So, the biggest surprise was that people who matched the avatar in just one trait, either gender or skin color, judged the rejection as less fair than people who matched on both traits or on neither. So, it must be one of those things where you're like, "Dude, you're kind of like me, bro. How'd you not hire me?" Or if they're totally different, you're like, "Yeah, fair. I don't deserve the job." Or if they're just like you, you're like, "You're just like me. If you don't think I deserve it, I don't deserve it." Do you guys even want like a humanlike face on the avatar when you're doing the interview or would you rather just I don't know have some kind of a cartoon character? I prefer to make it look a little less human. Honestly, I'd probably distracted by the little uncanny valley issues. All right, now it's time to dive down to the atomic scale and we're going to bring machine learning with us. So AI just found a hidden battery behavior that scientists have never seen before. And obviously batteries are essential for everything from electric grids to energy security to every little app runs on your phone.

[00:12:24]But one of the biggest problems is that battery materials are incredibly complicated at the atomic level. But this new AI had a pretty novel approach.

[00:12:32]Instead of treating complexity as a roadblock, some researchers at Lawrence Liverour National Laboratory used physics informed machine learning together with molecular dynamic simulations, which are basically just computer models that track how individual atoms move over time. And they end up creating this essentially atom by movie of sodium moving through hard carbon anodess. And then the model decided to learn that. Like, isn't that an interesting data set to learn from?

[00:13:01]It learned how those atom by atoms are like moving through hard carbon anodess which I I barely understand in my own head. But if it learns that the same way it can learn protein folding or the English language, it can do the same with this pattern. And it ended up classifying the way that it moved through these anodess into eight different patterns. And it showed that as carbon becomes denser and more sodium is added, ions can cluster together and get trapped in these tiny pores that can affect charging speeds and thermal safety. So it was able to avoid that, create a practical map for improving hard carbon designs. Boom. Better approach to lithium ion battery batteries like crazy. Researchers believe this approach could replace much of today's trial and error battery designs and speed up the search for better batteries across many different chemistries. And if that's not enough, AI is learning the clouds, or at least it's learning from clouds. Researchers are now forecasting solar power output by forecasting when cloud coverage is going to stop the sun from shining. But the sort of twist of it is is that the AI is learning from the power drops while watching the sky instead of like labeled clouds. So, it's weird. It's learning clouds, but it's also learning something bigger, something more connected to the entire environment and the power drops. So, at the core of the problem, solar panels have one big weakness, and that is clouds. A single cloud can slash a panel's output by tens or even hundreds of watts in seconds.

[00:14:33]And when thousands of panels are all connected to the grid, those sudden swings make balancing the electricity much harder. So to tackle that problem, researchers decided that instead of learning from thousands of manually labeled cloud images, it would watch the sky while recording the panels input and output at the exact same time. And then every sudden drop in electricity became a clue, helping the system learn which cloud patterns caused it. So, it's learning like the same way those billions of parameters are learning all these fascinating things about the human language and emotions. It's learning these fascinating combinations of just how clouds and environment just connect.

[00:15:12]It's tracking cloud movements. It's estimating where the sun is. It's probably calculating, you know, how shadows will fall and predicting how power will change over the next few minutes. And all of this is hidden deep inside that crazy neural network in ways, you know, it's like an alignment.

[00:15:28]like it just it's in there. It's hard to describe. It's a black box, but it's learned something special. And the point is that it works. The team tested it on real residential rooftops, not in laboratories. And over 92 days, they collected more than 122,000 synchronized observations of sky images and panel outputs. And compared with conventional forecasting methods, this this system, it's called Shadow Sense, cut average prediction error by almost one-third and detected more than 92% of sudden cloudreated power changes. All right, so at this point in the video, it feels like maybe AI really can do anything.

[00:16:04]And I sort of still think it kind of can, but we're also going to look at the flip side of that. Sadi Harley writes, "This article testing the limits of what's possible and what isn't with AI."

[00:16:14]Because you have to ask, is there a ceiling? Is there some sort of, you know, growth thing that's going to stop this? Like a petri dish, the bacteria is like, grow, grow, grow, and it's like, oh, we're at the edge of the petri dish and we ran out of food. Simple example, but in complex systems, there are certain ceilings like that also. And there may be some AI problems that just stay impossible, even if you feed the model infinite data. So researchers from University of Cambridge and UC Santa Barbara built a mathematical quote adversarial system that's designed to fool any AI algorithm. The goal wasn't to make AI look bad. It was to find the exact point where learning breaks down.

[00:16:51]And they found two main reasons why this happens. Sometimes an algorithm can't tell when it has enough data to make a trustworthy prediction. That's sort of I feel like humans kind of fail at that too. Like when do you know you know enough about something to make an accurate prediction about it? You know, we all have different baselines and we probably measure different, but also they have this problem. And other times the important patterns are hidden or too difficult to separate. That means collecting more data isn't always the answer. Even if you had unlimited amounts of data, they're just too tightly coupled. The team also showed why chaotic systems are especially difficult. In these systems, tiny differences at the start grow into completely different outcomes over time.

[00:17:33]Short-term predictions can still work, but long-term predictions become fundamentally unreliable. So, the takeaway is this. Understanding AI's limits may be just as important as improving its capabilities. So, more data isn't always enough. Chaos creates hard limits, and some problems are effectively unknowable, even for the amazing AI that we're building today.

[00:17:53]All right, now let's jump back to anthropic. It says that it has found something unusual inside of Claude. And I feel like it's worth really thinking about this. when a big AI lab says that its chatbot may have the building blocks of consciousness maybe unpack it a little bit more than when we've touched on it before. So the kind of question is when you kind of think in your own thoughts like do I know say where something's located and you're kind of in your own head trying to build confidence you don't say anything out loud but just you're kind of like okay I know that road I know I've been there before yes I'm confidence that's where it is so I'm going to give this person directions the question is about that internal system like does it just come out of claude like an automaton or a zombie that's just like d next word and then the next word just happens to be the location or does it sort know it internally. So they're trying to compare what Claude does to something called the global workspace theory. This is a major theory of consciousness that says the brain is kind of like a central hub where important information is shared across different mental processes. But that's where the debate begins. So the article points out that nobody agrees exactly what counts as a global workspace and that Claude's version works differently than a human brain probably would even if they both do have global workspaces. For example, human brains rely on recurring feedback loops and a process called ignition where certain brain activity is amplified and sustained over time. And as far as researchers know, COD doesn't do either of those things, but that doesn't necessarily mean that it doesn't have a global workspace. But even if it does actually have a global workspace, and they can prove it, or they can at least show the same sort of patterns, there's another question. And scientists argue over if organizing information that way is actually having this some sort of subjective inner experience the way we do. Just knowing that you have information that's sort of above other information in the same way that might match us. Does that actually feel like what it is to be something in the world?

[00:19:54]I don't know. But you know these findings don't necessarily prove that AI consciousness has arrived. But it's a great question like do global workspaces seem to be some sort of generalized thing that could be in different substrates but be the kind of architecture that leads to inner thoughts the subjective a feeling of being something and they nudge the conversation forward at least a little bit and it helps us ask that really important question if conscious AI ever becomes possible should we try to build it or build it at all like morally probably leave it alone or if you build it take responsibility for it which like a parent I guess over a child but even that doesn't make sense cuz you know someone would be building it not for its own sense of like hope you enjoy the world that's why I built you they'd build it so it could make them money anyway I want to leave you a little bit of an upper so let's talk about daydreaming so I'm going to try to daydream more you probably should too unless you're already hitting your daydream limits and you're max daydreamed out which you know there's I I have met a couple people who don't need any more daydreaming but for most of us it could sort of be the key to a better memory and it's also kind of about forgetting the right things to have that better memory and I was just like man you know in school how many people are like oh you should go daydream if you want to be smarter they should have so our brains don't just store memories they constantly sort them strengthening what's useful and getting rid of what isn't researchers have been borrowing that idea for a classic AI model called a hopefield network which works like associative memory show parts of something familiar and then it fills in the rest. Problem is that these networks waste space on false memories, the ones that mix together real ones almost like hallucinations. Earlier work introduced a daydreaming algorithm that lets the network learn new memories while clearing out false ones at the same time. That avoided a problem called catastrophic forgetting where too much cleanup starts to erase all the real memories. And it pushed storage from about 13 memories per 100 neurons to the theoretical maximum of one memory per neuron. Interesting. You can have one memory per neuron as the max memories.

[00:22:03]So if I remember right, I think the brain has 80 billion neurons. So 80 billion memories would be like our theoretical max. Interesting. But real world data is messy. Very bright or very dark images can look similar, making it hard for the network to tell what actually matters. So they have this new version. It is called centered daydreaming and it fixes that by comparing each image to the average instead of using raw pixel values that makes the meaningful differences stand out while still only using local learning rules that more closely resemble how biological neurons work.

[00:22:35]And I see parallels strengthen important memories, right? That's consolidation.

[00:22:40]Imagine what it's like to be more of a not a pessimist but a optimist. Yeah, that's what it's called. and you walk around just constantly strengthening good thoughts about the people around you. That's got to be the key to life, right? And I understand you might maybe live longer in some ways or provide certain types of value if you're worrying and you're always strengthening those worries cuz then you're not getting caught off guard by things, but there's also so much that can go wrong.

[00:23:04]It seems like, man, what a weird life when you just like constantly fill your brain with everything that can go wrong.

[00:23:09]So, strengthen important memories, weaken or discard unimportant or misleading memories. I mean, when your mind is wandering, you're just going for a walk, you're showering, a quick break, you know, you might connect ideas that seem unrelated, you might replay recent experiences, solve problems more creatively, strengthen autobiographical memories. That is the default mode network. And then in daydreaming, oh, I wonder are you're not really consolidating memories, but I feel like you're creatively connecting and then at night you're kind of locking that stuff in. I don't know. Another wild week, man. But just imagine, like, imagine you're reading 10 backtoback books without ever stopping to think about them. You'd probably remember less than if you actually just paused, reflected, connected the ideas, you know, like not just glossing over it. Daydreaming feels to me a little bit more like active thinking, and that's got to be part of it, too. But yeah, if you enjoyed this, you felt like some people in your world might enjoy it, hit that share button.

[00:24:08]Thanks for making it to the end of the video. If you want to support the channel, I've got a Patreon or a membership here on YouTube. And let's just keep taking everything day by day because it is getting faster and faster and faster. See you in the next video.

[00:24:21][music]