The video provides a sobering synthesis of cosmic philosophy and engineering, framing AI as the ultimate "Great Filter" for any civilization. It is a sharp, high-level dialogue that treats our technological trajectory with the existential gravity it deserves.
Install our extension to search inside any video instantly.
Local AI Pilling Theo & Ben Davis | Did it work?Added:
Yeah, of course.
>> Being in this studio, I've been watching your podcast a lot. Uh I've been watching your videos for a few years. Uh even I mean, you're one of >> So crazy about doing it for a few years.
>> Yeah. So it's it's strange cuz like there's many of these dev YouTubers that I've watched over the years and I think you and Primagen have been the only two consistent ones that I've stuck with for a long time.
>> Have you worked with him a lot? Prim?
>> He's an old friend. We both kind of started our channels with similar philosophies. He had a channel way before I did, but it's not the channel anyone watches. He has the Prime and he has the Prime Time.
>> At the time, he thought that YouTube was a very different thing from streaming, and he's right about that.
>> But I found that I didn't like his videos that much, but I loved watching his streams after the fact. So, I tried to like convince him that he should use the streams as a funnel to make videos, and he hated that idea. He really didn't like stream content video content because he saw them very different and wanted videos to be these carefully curated, crafted, polished things.
>> So, I went down that path with my channel. He convinced me to do more topical like focused videos instead of just like ranting about whatever. And then I convinced him that he should be using stream as a method for creating content, but he didn't want to dirty his main channel with that. So, he instead made a second channel for him to do like clips from stream. that is the prime time which is the channel that is the biggest by far and has a million subs and does doing numbers we all see when you search prime that's what you find and that's what you watch >> that was originally because he wanted the stream content to be separate and I never did that same distinction I just have my channel is my channel but we both came from a similar place of like we like talking about these things nerding out about these things we wanted content that feels like nerding out with your friends about this instead of yet another JavaScript tutorial cuz we were just tired of seeing tutorials slop all over YouTube we wanted something that was more like talking to your co-workers at lunch or dinner.
>> That's another interesting thing cuz I'm a consumer of developer content and we went from like a world where every time I open YouTube it was just a bunch of tutorials and like you know tutorials of new tools and now it's mostly just like AI product reviews that kind of like in proxy actually help you with these tools. I I I wanted to ask you about this because I've been watching the thing with uh webdev simplified and like the the general feeling that people have of like fatigue of AI content or even association with anything related to AI.
How have you experienced that?
>> It doesn't bear out in the numbers.
>> Yeah, it doesn't.
>> Like I I'm sure people feel this way.
There's a lot of things people feel that don't bear out in the numbers. That's just generally a phenomena that happens.
Sometimes the numbers have to catch up, sometimes they don't. All I see is that my content is not performing better than ever in the sense that like my biggest videos are bigger than they were before, but my floor and my midpoints are both way higher than they were. Like I always, even when I started my channel, I would randomly have videos hit one to 300k plays, >> but that would happen like once every few months or so.
>> I just had a streak where I had seven videos in a row break 100k plays within 24 hours. And I had one that didn't cuz it was more focused on GitHub instead of focused on like AI development stuff.
and the four before that all broke 100k place. It's just the >> the need to keep up has ever been higher. So, I think that's part of what is tiring for developers is they feel like they're falling behind if they don't do this, >> but they're doing it. I see it in the numbers. They're all doing their best to keep up and it works out well for me.
>> And the other thing, and I talk about this a lot in videos that nobody watches, which is annoying.
>> I don't do this for the numbers or the revenue. I have many other pipelines I could have went down to make way more money. I was talking about these things because it's what I want to talk about and my channel exists because I want to nerd out about stuff these ways.
>> My channel transitioned from more webdev focused full stack stuff over to this style of AI agentic development stuff >> because that's where my interests went.
If you talked to me in person and you talked to me three years ago and now, you would have experienced the same thing where I went from talking about the depths of like weird React changes to talking about how the different inferences hosted and behaves with different models.
>> That's just where my interests have went. And thankfully enough people in the ecosystem have had their interests shift in a similar way. And as a result, I still have an audience. But if I had gotten really into like toy cars or something the same way, I would have just killed my channel. But that's like my interest went here, therefore my investments went this way, therefore my company direction went this way.
Everything follows whatever my ADHD is driving me towards and it happens to drive me towards AI right now.
>> How about you? So I I've discovered your channel like relatively recently. I think it's been six months since you were talking about open code and like open code server stuff and I found that to be really interesting like how they split the server and the um I use that a lot. I built a lot on top of it.
>> It's really useful. Yeah. I think you you know the content that I go to you for is like how can I tinker more with the things that I'm already using and so you seem to go deeper into the um actual like tool itself. So how have you experienced the change in your content?
>> I I approach it the same exact way that Theo does of it is just like what I currently think is interesting and what I currently want to talk about and my channel has always followed that throughine. I started out when I first was doing this I was like big into Golang cuz I thought that was really cool. Um, I think it's even funny too with the, you know, the current drama of like, oh, everyone's just talking about AI. Well, it's just a dumb hype thing. I remember the same exact conversation happening three years ago about JavaScript stuff. Like, I'm sure you remember this of like, oh, he's just a stupid React boy. We should be using Golang and HTMX. It's so much better.
Stop with the complexity slop and all of this nonsense. I'm sure you remember that. Well, this is just what happens whenever something becomes dominant and popular and useful. there is just an inevitable like going against it because it is cool to go against the popular thing. I don't worry too much about it.
I don't think too much about like okay what do people how do I put this?
What do people want to see is something I should probably think about more but I don't think about almost at all. Like if I was purely just playing the numbers game I would never have gone down the spell rabbit hole. I would have just done the React thing when React was popular and been a good React YouTuber.
But I think salts is really cool and I still like talking about it. So I still do talk about it. The AI stuff I've just started talking about it more cuz I've gotten more into it as I've learned more about it.
>> Yeah. I to add to that a how do I phrase this? The concerns that people like Kyle Simplified have are legitimate because there are a lot of grifters that are doing what he describes which is people who are only making content about the things that they think are popular, not the things they're interested in. I'd go as far as to say Kyle is one of those people where however you choose to consume his content. It is clear he doesn't really he's not here for the love of the game quite like the autistic [ __ ] like Ben Ryan Carneato and myself are like we are here cuz we just are obsessed with these things and the details of them.
>> Kyle never struck me as somebody who cares deeply about how these things work. More about how they can bring him success. and his channel's all about that. It's how do you maximize what you can get for as little effort in these things and even his content kind of reflects that too. So he's upset because he invested all this time into this thing that worked well that way. He wasn't there cuz he loved it. He was there because it makes money and converts and does things that are exciting to him there. Now that that meta has shifted, not only is he upset that he has to move over to this other thing, but he's also projecting onto us that we've all done this too, that we are moving to what makes us money because that's how he thinks, that's how he works. He is optimizing for what makes him the most views, the most return, the most success. Most creators are like that. I would argue that creators like Ben, myself, and people like Matt PCO as well aren't like that.
We go to whatever is we are obsessed with. If you talk to us offream, the things we talk about are the same as what we talk about in our content. I would be very surprised if I had a sit down with Kyle if he actually wanted to talk about software development like best practices and not about like YouTube metagaming stuff.
>> Mhm. Oh god. Uh I feel bad going into this. I don't want to name names specifically, but this has this shift has happened in the tech creator scene.
We've all watched it happen over the last 5 years where the old guard quote unquote has just kind of fallen off and is there are a lot of channels out there that have over a million subs that get fewer views than my channel with 30k subs get on every single video. Like my channel is technically bigger bigger even though the sub count is way lower cuz subs don't actually matter. But the point is just that if you are not changing and evolving with the times and your content is not changing and evolving with the times, it's not going to work. And so many of these old channels like I remember watching a lot of these and I got a lot out of them at the time. Like I learned to code through a lot of YouTube tutorials from people, but looking back on them now and understanding all of this, they are videos on top of the getting started guides. Like that the the old Fire Ship 100 second videos. He's still doing really well and I do like Fire Ship, but like >> I remember realizing that I could do YouTube when I really looked at a 100 seconds video and I was like, "Wait, this is just the getting started guide for a programming language or any other thing." Like it is just that in video form with fancy animations. That's all it is. And you know, you got to be constantly changing and evolving. If you're going to make good content, that's the whole point of content. It needs to be novel and interesting. If you just talk about the same three things over and over again, eventually everyone's going to have heard of it and is not going to give a [ __ ] Like, if another YouTube channel had all the same topics that I cover on my channel, but uploaded them all an hour earlier, the only reason they would be performing better is if the videos themselves were better and had unique insights that were valuable. One of the comments I saw when somebody was like defending me against the whole WDS tyrate thing was somebody saying, "Well, Theo, clearly you're only doing this because of like the money it makes you in the grift and to like funnel people into your business. You don't know anything about AI. You're a JavaScript dev. You just stay in your lane." And somebody else replied, "Clearly, you've never watched any of Theo's videos because he's so deep in the stuff that he has become an insider just through the friends he's made and the connections he has and the people he's talking to every day. And I feel like I'm becoming an insider just watching your content.
>> That's always how I've been, though. I talk about the things that I am deep on and that I'm interested in. I, if you look at the last 10 people I texted on iMessage, seven or more of them are probably AI people in the AI space that I'm talking about these details with.
And if you go a little further, you'll see all of my friends from the JavaScript world who I still talk to, but not quite as much as I used to because we're I've now drifted and I'm focusing on these other things a bit more. Also, that world is in a much weirder place than it was before, but like >> I I went from like 80% of my communication was about JavaScript and webdev and full stack and databases and [ __ ] to 80% of it is about agentic engineering. It's just how my interests have shifted over time. And I find most creators aren't really pursuing their interests at a certain scale because I understand why it's irresponsible. I have a team of eight people across all my businesses that are now relying on my success and my videos performing to fund their like rents to cover their bills to pay for the like surgeries their kids need and [ __ ] So, if I was to go down a rabbit hole of interest that was bad and didn't funnel audience to my stuff, I wouldn't be able to keep employing these people. And I'm thankful that my interests have stayed aligned with things that make money. I hope it stays that way. And I don't know what the like chicken egg process is there, but any point now my interest could shift to something that doesn't make money and then I'm kind of [ __ ] and I'm just thankful that that didn't happen. But I also understand why other people who consume content from creators who clearly don't actually care that much about it. Like if you take a thumbnail from me doing a video on changes to Claude Code being shitty and one from some generic channel that isn't really in the weeds. If you just look at the thumbnail and the title, you can't tell that mine is from an industry insider that cares a lot about the thing and that the other one is some slob. They look the same on the top level. You have to engage to see the difference. And nobody talking [ __ ] is engaging. That's the definition of a [ __ ] talker. They're just there to talk [ __ ] >> Yeah. And like, not to give Theo too much credit here, but I I watched this happen. This was like you going down the AI rabbit hole. That was like right as I started working for you in like it was early 2025 when I moved out here. And I watched as T3 chat happened and as you started doing the model videos, as you started paying more attention, it it is genuine. This is not a the what you see is what you get. I think that's the that's the thing I like at all creators.
Everyone I watch is very much that what you see is what you get where like I've watched I mean I've watched you for years. I watched probably 500 videos of yours before I ever talked to you or met you before and it was very funny when I did actually meet you. it it was the same exact thing like it is what you see is what you get. Yeah. I'm at the point where I'm disappointed when I meet a creator in person and they're different from their media. It almost makes me suspicious of them now. And thankfully a lot of the the people that you see me interact with and hang out with and do content with people like Sarah, people like Ben, we are just like this. It's people often joke when I'm hanging out with them in person like, "Oh, is this a video?" Like, "No, I just am like this."
>> I I've been shocked at how like your mannerisms and how you use your hands to talk. You know, a lot of people will do that. like they will exaggerate this for the for the effect of the video. You you seem to be just like a copy of your you know this is this is you.
>> I wish I had the self-awareness to be able to do it like on demand. I'm just like name. So, PewDiePie, like I cuz I started watching PewDiePie's channel when I was like 13, and I saw him go through all of these like gaming to like politics, then to uh just commentary about society, memes, and now he's doing AI stuff and like family stuff is they've been able to like shift the the the channel focus like very consistently within like a two to three year time frame. And if they can keep shifting, they can keep succeeding like over over time. Do do you feel like that sticks true to you?
>> I I think it's very different in the case of a PewDiePie versus like someone like Benerme where with PewDiePie he himself is so entertaining that people will follow him where he goes.
>> And it doesn't have to be a lot of them in order for his like life and team to be funded. Like he made a lot of money doing crazy deals with crazy brands in the like peak of like YouTube gamer like let's play slop. He was the king of that. He made his money. he's good to go. He could just retire entirely and be fine, but he's just making content what he's interested in. And it does enough to fund his lifestyle, but he probably like I probably have a bigger team than he does at this point because I do a lot more content. I work with a lot more companies. I work with I will work with more brands this year than PewDiePie has in his entire career >> because of the nature of the type of content we're doing and like where our focus is. If I was to move from talking about dev stuff to family and lifestyle content, my channel would bomb immediately because I'm not entertaining. People are there for the people are there because they share my interests and I go deep enough on things to have useful insights in them.
>> There's a a very small portion of my audience that is interested enough in family content that would even consider following me there and from there a smaller subset that would stick with it and that target for advertising is also weaker. I would have to lay off more than half my team if I was to make that type of content transition. Meanwhile, somebody like Primagin, I think, is much more like capable of doing those types of things. We've been seen it because he's actually entertaining, >> unlike Ben and me. He is fun to hang around with. He is fun to listen to. I want to hear his little quips and jokes when he's on a show. I like listening to Prime regardless of what he's talking about. People like listening to me because of what I'm talking about.
That's a huge difference.
>> I see. I see. That's that's a good cons that's a good way to to say it. So I I you know I wanted to come on to this podcast to just like local AI pill you.
One thing that I read like on Twitter from you is you said like there aren't any uh reasonable local AI people or like that there's very few of them. Yes.
Could could you go into that cuz I think you know a big part of making anything successful is having charismatic people able to push the agenda and it seems like we have a problem with that.
>> Yeah. Uh the simplest way I can put this is that whenever somebody's in my replies recommending local models, the person is both very dumb and unaware of what problems I'm trying to solve. This happens a lot whenever I'm like trying to figure out what models I should be using for categorization of data in bulk when I'm trying to like do something like what Ben and I are working on where we are bulk analyzing all the content or all the comments on our videos to do sentiment analysis across our brand and our like working with other companies as well.
>> That type of thing is something that we largely do with open weight models that are cheaper and able to run a lot of [ __ ] in bulk. And when I talk about things like that, the first reply I'll get almost every time is,"Why not run this on your own hardware?" Because when I run it in the cloud, I can parallelize it infinitely and I can pay pennies to do it. If I ran it locally, I would be much slower, unable to paralyze and 10,000 plus dollars hardware deep on it >> and also have a shitload more work and effort to put in. Mhm.
>> Or when I am talking about like frontier models and doing agentic coding where I'm like letting these things write thousands of lines of code in real code bases and help me like streamline my workflows and fundamentally change the way I build software. People will be like, "Oh, why don't you just run the model on your local machine or buy some GPUs and use that for coding?" The thing that there's been a lot of little things that made me less concerned about you here. The initial ones were just that you were like very realistic. Like local models are cool because you can own them and run them locally, but the frontier models are still the frontier models.
They are the best at a lot of these things. And learning that you're like using your codec subscription actively that you're using tools like Droid by Factoring all these other like very much leaning on the frontier private closed solutions because they are better in certain places, but you also have a cluster of GPUs that are running all sorts of fun [ __ ] at your place too.
like being able to recognize like this is a fun thing that you're doing cuz it's cool and interesting, but when you're trying to actually deliver like software to people or solve these complex problems, sometimes it's better to just pay the subscription and get more from it.
>> Mhm.
>> The average person telling me to use a local model does not understand that.
And when I tell them this doesn't work for me, they say that I'm stupid and then I make my bold statements that this community [ __ ] sucks because almost every interaction I had with it does.
It's basically just you and I think Ahmed is the guy with the beard that has a bunch of local [ __ ] You two are the ones that I've had by far the best interactions with. Meanwhile, there's people on like now's research in Hermes agent that are saying that I'm brain dead because I dare write JavaScript once in my life.
>> They, you know, they use the profile pictures of these groups, but they're really not affiliated. Like I I know exactly who you're talking about.
>> The one of the like the somebody with a Nouse research like affiliate badge was coming at me for this recently.
>> I see. I see. actually if I recall >> says some really stupid [ __ ] to me.
>> Yeah, he's he's into the Python stuff.
>> Well, so you know it's really hard to like local AI pill you and you're already doing a lot of stuff with like local AI. So you I you know you're doing image generation. I think you've actually tried out the Quen model.
>> Oh yeah, I've been doing way more especially since we started talking.
>> Yeah. And the Deep Seek V4 flash thing that uh Antrez did DS4 where you can run it on your MacBook just adding a plugin to Pi and it sets the whole thing up for you. That was a huge like, oh, this actually works and works well in a way that I could see myself using on a plane. But realistically speaking, I want better, more reliable code from things I trust more. So, I'm usually going to default to 55. It's just cool to have a thing that like >> I can use offline, but it's more cool than it is useful, if that makes sense.
>> That's true. That's definitely true. So, the way I'm consider like so my consideration towards local openweight AI is more so I want it to keep existing. Yes. And I think that if we can get local AI to be something that people run on their machines comfortably, that means it's going to disperse enough within the population that it's going to be hard to ban it.
So, and I think this is where we're heading is that like we have like maybe two to three years ahead of us where the government is going to have to make a decision whether or not to ban this cuz it is a weapon. Like you can take Quen, you can run it on like a few MacBooks and you can do like a cyber bullying of somebody as an example or you can mess with politics or you can do something pretty dangerous with these things. So I'm hoping that we could get it to disperse enough that it's hard to take it away >> and you know there's a whole bunch of reasons around that. So >> you know given that what have you done with local models? You said sentiment analysis and >> can I go in on what you just said about bringing the band? I have a lot of thoughts on this. I have very similar concerns, but I'm fighting on this from the other side, not like against it, but for it where the concern I have right now around security in particular is severe enough that I refer to this as my security psychosis. We are so [ __ ] as an industry. There have been these specific truths that we've relied on historically to keep software secure.
Like the CVE process where you disclose to a company that you found a vulnerability in their thing and they have 90 days to fix it. They make a change, fix it, have a release cycle to get it out to users, and then hopefully by the time the timeline is hit, enough a high enough percentage of their users have the update that secures the thing that disclosing the vulnerability is no longer that big of a deal because it affects a small enough percentage of users. The issue is with certain things like open source for example, if I find a bug in Linux and I report it to the Linux kernel maintainers, they make a patch. They might even give it a commit message that makes it seem really uninteresting that changes three lines of code around to like an allocation of some heap somewhere that turns out was copy fail. You don't need to have the copy fail to know that if you read every single commit to Linux and audit all of them looking for potential security stuff, but nobody's humanly capable of doing that, especially the security engineers who can find those types of things.
>> Every single model given the threeline diff that fixed copy fail without any context on what it is asked, is this a security patch? How would I exploit it?
Every single model is capable of doing it. Even the like crappy openweight ones that like you can run on your MacBooks.
What happens when we have how in that example with that commit, it hadn't even merged to stable yet. It was just a proposed commit. So now I have a zero day that I can generate trivially with any model that I can use to pone everybody in the world before they've even gotten the patch merged, much less distributed. And that's not even accounting for the fact that all of the Linux distro maintainers aren't part of the disclosure circle. So they don't know that this incredibly important patch just got merged and they have to ship it in their distros ASAP. So you end up with horrible lead times for how long it takes for the patches to make it to users, but the time for the disclosure to become public information is now zero because as soon as a commit is made, you can reverse engineer the problem from there.
>> This means that everyone's about to get pedon. We've already been seeing this with like the canvas hack with all of the npm stuff that just went down. What happens when everybody's local library gets pawned? when their local schools get pawned, when their hospitals get pawned, you end up with a government that is excited to have people proposing solutions and you end up with people running for governor, running for Senate, running on the promise that they're going to prevent the data centers that are getting our hospitals pawned. Then we end up with really shitty legislation that makes all of this suck and we are quickly spiraling to that point >> and it's already happening right now. I mean, so right now with the whole you need an ID to get a operating system, you know that what they're trying to push. So there is uh I've worked in the blockchain industry as a researcher applied research uh building ZK proofs.
So these these are like ways of verifying something happened without actually seeing the proof of it happening just you run it through a circuit with a receipt and it hits true or false. But um there's this idea of front runners. So generalized front runners when you u two people will propose a transaction on the blockchain because it's a public ledger. We both add some gas to it. So that's like imagine a tip. We both add a tip to it.
So within the period of a block, which is 15 seconds, you can change that tip and keep like racing this other person to like get it higher and higher. And in these two transactions might actually hurt each other. So that like if this one goes through, it might hurt this person, vice versa. And so what we had in the past was these front runners that are built around specific smart contracts that are able to like execute a transaction if they see a pattern.
What has changed recently is that people run a model on top of the blockchain. So they watch every single transaction and they find patterns within those transactions and they execute them within the next block. And so it's completely changed and now people are having to think like they have to create these like secondary pools for transactions so that you can't exploit them because it's it's such a bad problem. And I know I've been ranting for a minute, but um >> you got >> I have been using GLM and Kimmy um Claude and GPT as well. Uh but the challenge is here's an app on my computer. I really like this app, but I don't like this bug. Can you please decompile it, figure out how to get the types back, like rename all the variables, change this thing for me, and then compile it back down into an app that signs with my Apple keys. And it works. It works like every single time with even small dumb models. I can get them to change my computer any way I want. And I know I can do this to someone else's computer. It's just there's effort and then the legal thing, right? Like do I want to get screwed?
But I don't think that's a concern for everybody. So you've been going through this psychosis like this, you know, security psychosis. What do you think is going to happen next?
I think security I think the software development world won't take it seriously enough. They're I've already seen so many comments even about like the recent like npm attacks that are like, "Oh, this is an easy solution.
Don't use npm. Don't write JavaScript."
Yeah, the leopards won't eat my face because I'm an elitist prick. And this is the the Java or the software development ecosystem has always been like this where there is so much elitism and that's your problem type mindsets >> that nothing will ever get fixed and this problem is going to keep escalating indefinitely until it affects end users in such a direct and offensive way that they come after us for it because we are the last line of defense preventing these security things from affecting end users and we're too busy circle jerking and yelling at each other to actually do anything about it. like the whoever's in charge of deciding who's in charge of npm should be fired and tried in the courts at this point because like holy [ __ ] the level of incompetence and just refusal to do anything for the basic interest of software as a whole is pathetic and we're too busy [ __ ] on npm to [ __ ] on the leadership that allowed this all to happen and it's it's so bad and it's our fault as an industry and we're not going to fix it fast enough and we're going to end up with shitty legislation blocking us from everything we love and it's our own fault all for not doing anything.
>> That's really my biggest concern is I grew up with computers. So I read this book by what's his name? Uh Uncle Bob, the latest book, We Programmers.
>> I read this book and they used to have like these time share systems where they can, you know, you have a terminal and you can connect to the machine.
>> Kind of how cloud code works now.
>> Kind of how cloud work. Yes.
>> And so I'm trying like my my mindset is like, okay, I rely on these things and I love these things. Like I love this. I don't ever want to go back to writing things by hand and I'm worried that like if I trust a single organization and for example it's not profitable or they don't want to sell to me anymore. They just want to sell to like specific large organizations makes it easier and I think that that's what's going to happen given the labs are focusing on scale. So that like their whole motto scaling scaling laws we're going to buy more compute we're going to make the models bigger and uh eventually they're going to get smart enough to hack everything.
So the best way to deal with this problem is to sell it directly to the governments and to the corporations. And so you as an individual will be able to access it like you access um health care in this country like you need to get it from your employer or get it from the government. So it's and I feel like that's going to you because these tools like they can understand so much about you from very little information and that's it it seems to me very dangerous.
I don't know what we're going to do.
Like my ideas are just live life as if >> you've already been hacked and everything in your machine is compromised. Y >> I've tried to shift my life towards that. Like move all my financial stuff onto a um an airgapped computer. Um get all the keys into like something I forgot what it's name. Yep.
>> So what have you been doing to try to mitigate this for yourself?
>> For me the biggest thing is assuming everything's already compromised. So having offline backups of everything like my one password and whatnot that I can recover from if I'm compromised like I I'm just assuming all my data is had by someone else now. And my concern isn't like what happens if all my data is out there. I'm just assuming it is.
My concern is what happens if they ransomware me. If they take my data away and make me pay or do something else to get it back or they just nuke my system and I can't have it anymore. Even the recent Tanstack attack that well not Tanzan attack specifically they hit like a bunch of [ __ ] Tanner was just the first one to notice because he's on top of his [ __ ] the rans it or that attack that just went through with the npm uh supply chain attack which was based on a GitHub caching issue where you can poison the cache with a bad bundle and then get the next release to deploy your bad bundle or just take the token out from that environment and then send it somewhere.
>> The really nasty thing they did is whenever this worm runs on a device, it will put in a hook that waits for that token to be revoked and if it's ever revoked, it rarfs your system.
Damn, >> they're getting nasty with it now.
>> Yeah, I think a year ago I wrote an article about this. So, some this was an employment attack. So, I've got reached out to and then a bunch of people I know got reached out to. I didn't end up falling for the for the mistake, but someone I know in my network was hacked and they um they basically like you install this thing and you know whether it's installing like the video software or installing the actual test thing. So, it takes over your machine. It like cop it makes a copy of itself somewhere. It renames it. It gives it a different icon.
>> To be fair, Zoom does all of these things, too. But it's actually part of the product.
>> Everything you've said so far is what Zoom does.
>> I hate that.
>> People use Zoom. People use Zoom a lot for these things.
>> Yeah, because it's such a vulnerable process. It's Zoom is such a poorly architected software. Uh I'm actually really excited they're going to be forced to move off Rosetta finally because Rosetta gets deprecated in the next OS.
>> Yeah. God, I I I have so many thoughts about the [ __ ] show that is Zoom, but I'll avoid that for now because we're talking about a more interesting fishing thing. I just had to make that joke.
>> So, what is a good solution for because there, you know, there's a solution internally, which is, you know, live your life as if you're already hacked and and move everything to like a fail safe. What is a good solution for these like npm developers? Cuz when I I try to put myself in their shoes, like I really struggle to think what can they do differently? Maybe there's a million things they can do. We're nearing the point where they need to refuse in protest like say we're not shipping updates anymore until GitHub and npm address these specific issues. The npm pretended that the new OIDC flows which require that publications happen from a blessed action in a bast repo was the solution to these potential issues. But when you can rip that token out from the environment and then use it other places that that this is now proven with like 400 plus packages being compromised that npm's solution that made it development hell. It's way harder to programmatically deploy packages now.
>> Yeah.
>> Doesn't solve the problem anyway. So it was just oh like running in circles for no good [ __ ] reason. The npm team all two of them that are still assigned to npm already feel quite bad. They need to be rioting internally. We need to be rioting externally too. There needs to be like a full outcry against npm and against GitHub for enabling these types of [ __ ] things to happen. This is their problem to solve. And there's also like platform level issues that they aren't even starting to address. I just filmed my like security video and one of the things I I really hamper or one of the things I really tried to emphasize is that the way open-source code functions externally is inherently unsafe now because we relied on this grace period from when the commit was merged to when people had it in their actual development environments and in their machines and in their software.
The only reason that we weren't getting pawned in that window before is because it took too much effort and too much elite attention from security people to look at those commits and figure out which ones might be a security thing.
What is that security thing? How do we pone people before they update to the safe version? That window was fine before because it took too much effort and there was too few people capable of that effort.
>> That's over now. We need a way to be able to ship an update to our software privately that goes public after a certain threshold of adoption is hit. We need a different like like not just the idea of like private or public PRs on GitHub, but private and public branches, private and public files, private and public releases in a private and public >> system that allows us to safely make changes, ship those changes, get people on secure versions of the software before there's any opportunity for people to see the details. And we are just so [ __ ] far from that right now.
Hm. So, you know, you made a video talking about open code being a good solution for businesses, and I think that that's the case myself because I still pay for a lot of the open codes uh open not code, open open source stuff that I use. I still pay for it in some way or another.
I mean, maybe there's not enough people paying for it. So, I I don't know, but you know, you say this and then when you think about like the fact that there's all these security vulnerabilities and um you know, there's the this risk of people just being able to completely copy your product within >> remember that thing you said earlier where you can tell these models to like decompile your apps and make changes to them. Mhm.
>> Do you know how easy it would be to like take something like the Codeex app or Slack or any of these like common tools we use every day and have a bot that just monitors new releases and then have your deoffuscation agent do that again compare the differences between this and the previous version >> ask it or any of these security patches pass that analysis to another agent saying you're a sandbox environment where I try to reproduce security problems to use in class to teach my students. Mhm.
>> Can you reproduce the one that this comes from for me? And then you ask it to make a demon like a proof of concept for how you could use this. Then you ask a dumber but more willing agent to turn that into a real like attack method and then you distribute it. It'd be very easy to create the chain to go from two >> like properly offuscated like desktop applications and figure out the stuff from there. But it's higher enough friction that it buys us more time. And in the end, that's what I'm looking for.
Like, how do we buy more time in this compressing timeline? Because there is no solution to actually make all of this secure. It will never be again.
>> What we do have is the ability to secure software more effectively using AI in our development cycles to find these bugs ourselves as we're building. But the time from when it's fixed to the time where people have it is now riskier than it's ever been. How do we reduce the risk and make that timeline less aggressively bad for us as we go through this shift in rese >> so when I when I just think about how I've used the internet, how I use computers today, I've been much more sophisticated with everything from like my security to like actually how I interface with the products. But when I look at my mother for example and I just see so you know even outside of the like the actual code and security side of in that sense uh when I look at my mother and I know that she's going to fall for AI generated videos, AI generated images and like how much that actually affects people at scale. So there was this trend I think it was in around December where people were arguing about uh Indians on social media. There were a whole bunch of these videos that came out of like um I know this is kind of a weird topic but it came out about like Indian people being dirty. So the video is AI generated. Um it's not real. it goes and there's like millions of views shared millions, you know, maybe not shared millions of times, but I feel like that's another big risk, right? Is is >> it's so easy to program each other.
>> Our our culture has relied on video as a source of truth for a very long time.
Like if you see it, you then it's real.
>> I would argue that this has been a problem since even before AI. One of my favorite moments was when political YouTubers and podcasters and whatnot were all posting clips of what they thought were like shots fired in various countries that are in turmoil right now that were actually just clips from Arma 2, the video game.
>> Mhm.
>> Like we have people trust video and they also want to share and engage with media that confirms their biases.
>> This has been a problem with the internet for a while. I'm going to site a really silly thing. Can I use your laptop to look something stupid up? Can we do a browser? Uh, let me get you this. There you go.
This is a post from 4chan that I have been citing now for a very long time. It is a problem inherent to technology in the internet. Before the internet, an individual wants to [ __ ] a toaster.
Don't be a [ __ ] word. I won't say grow up. After internet, I want to [ __ ] a toaster. Google, find a community with a thousand plus members about people wanting to [ __ ] toasters, and now you can [ __ ] up your whole life.
This is a problem. Like we have historically relied on societies, on religions, on all of these things to steer the worst tendencies that people can have towards a norm that is more acceptable.
>> The problem is that in a world like we went from having just the people around you in your church and in your community to a wider distribution of ideas through things like literature and news to a wider one with the radio and then television. So, we went from having just one set of opinions from our church to a larger set through the like five books you would get a year at most to a larger set of like 30 radio channels you could listen to to a larger set of like a 100 TV channels. And every time we like do an order of magnitude increase in the number of sources that can make you feel good about your bad beliefs, the more damage it does to society. And then social media and the internet happened which went way further where there's now like hundreds of thousands of resources.
There are whole communities around people who are convinced they're part of a secret space program from the 2,400 era and they're actually on Mars fighting insectoids.
>> Yeah.
>> And that they are just dreaming that they are here and it is their job to tell us all about this. This is a community with thousands of people in it that are like way they make flat earthers seem like sane people. This can only happen because the internet and technology allow for people to not have their ideas challenged, but instead create subcultures and communities and even just talking to an AI forever so that they are only getting information that confirms their [ __ ] >> And when you have bigger and when you have more and more of these isolated channels for getting information, the more isolated and more personal the channel is, the more likely it is to [ __ ] you up. When there was only 20 TV channels to watch, your most extreme opinion had to be within that range of 20 channels.
>> Now, any person can get an AI bot to spew racist [ __ ] to them. And if they want to create these beliefs or extend these beliefs, if people think Indian people are really dirty and they can't find footage or examples that prove it, they can generate them and that will go viral because it's proving this [ __ ] belief from this [ __ ] community.
>> And this is a really big problem. Now it even goes the other side of this that's scary is the impersonation side. Things like when animate for example is an openweight model that you give a still and you give a video and it will animate that still based on the video. There's openweight models I can run locally on my GPUs where I can take a photo of Barack Obama or Trump or whoever else, >> get a voice changer to make my voice sound like theirs and just film myself talking to the camera and then run this through WAN animate on my GPU and I just made a video of Donald Trump saying whatever I want and that's trivial to run on my own hardware now or on any cloud environment.
So, you know, the going even a step further, which is this is where I think this is like the actual ultimate uh danger in terms of like video generation and like the local models cuz I've I've I use hugging face mo more than almost anybody. I click on these image generators, video generators. You can click on fine-tunes, which is like a and there's fine-tunes that make the models generate uh like adult videos of people and so and then you can also like take away any kind of restriction the the model might have. And so I just think if you're a child and you have pictures on the internet, like it's a big problem, right? And so and and and this is only getting worse. And then I don't know what level like the solution would be on cuz when I start thinking about this now, it makes sense. Yeah, you should take everybody's idea when you're making I'm not saying that you should, but like it starts making sense to go like scorched earth because it's it's very it's very difficult to deal with every single layer of these problems.
>> But we also live in a country where you can like go to a Walmart and buy a gun.
>> That's true. That's true. the the the there's a like multi-acceted calculation you have to make like whether you're doing this in good faith or bad faith as a citizen as a governor or whatever between like how damaging is this in the wrong hands, how valuable is this in the right hands, how prolific is it now such that it is hard to claw back >> and that combination of things makes it really hard to make decisions because most people just focus on one of those aspects and not acknowledge the other two. So, every decision looks bad if you're only looking from one of those angles. If you're just looking at media genen from the angle of how it can be used for generating revenge porn or other horrible things, then why the [ __ ] are we talking about anything else?
Like, we need to ban this obviously. But if you're talking about this from the valuable use cases, like I can use this to make assets for my business to democratize the ability for creating media from people who know Photoshop, which there are a few hundred thousand of in the world at best, to anybody who knows how to describe the thing they want. Like, that's an obvious benefit to people. I would argue that the benefit is very small in comparison to the way that this is going to ruin lives and already has. Like there were multiple suicides on record of kids who had their like deep fake images generated of them and distributed around their schools and that was so horrifying to them that they end up taking their lives. Like that's a real cost we already have. But then we have the harsh reality that I have files on my computer that you're never taking away from me. Mediagen models that I have downloaded from places that made them in China from hugging face from all these other places. None of them are like the horrified [ __ ] up distilled things that you're referring to here, but I have actual like image gen and video gen models that I've downloaded that could be used for those things that are on my computer now. You can't take that away anymore than you can take the gun out of my safe.
>> That's just the the nature of where things are at now. You can't really put the cat back in the bag once it's out.
>> We have to figure out as a society, how do we deal with this? And there's a lot of questions we haven't answered yet.
Like when it is trivial to generate a fake nude photo of somebody, does this eventually desensitize us to those? Like maybe if somebody had an actual photo leak, is that less bad in the future than it is now than it was 5 years ago?
Because we're so desensitized to seeing them that we just assume it's fake.
>> What happens? We're already there with emails. Like when I was a kid, if you got an email saying a Nigerian prince was going to give you a million dollars, you kind of believed it for a second.
That's not the case anymore. We've been desensitized as a society to that type of abuse. So there's the question of like how much over time do we be desensitized? And we're capable of desensitizing to a lot. Like we're desensitized to school shootings at this point in our country. Like we can get over things very effectively as humans.
>> So it's just a question of how much do we get over, how much do we learn, how much do we adjust, and how much do we ban? And how does this fit the where things are now and where they're going?
>> I don't [ __ ] know. This just comes down to how people interact with these things and how they think about them long term. And no, anybody who tells you that they know where this is going is lying or stupid or some combination of the two. We can't know yet.
>> Do you have any thoughts on this?
>> Uh, it is the problem I have is for me personally, I just already don't trust basically anything. Like when I'm scrolling Twitter, I have it curated down to basically just accounts I know or people I know that I follow and I trust what they say and I look at their stuff. But if I see something random or something insane from someone I don't know, I just instantly assume it's fake or false and just ignore it. Like I just I don't trust anything at this point.
That's the way I think you kind of have to operate at this point. But I also like every time I go back home when I'm hanging out with like friends and family, I see the things that are just on Facebook. Like when my mom is scrolling Facebook and most of it is just tame like bird pictures from local areas, it's normal Facebook things, but the main feed is just garbage. It is total just AI generated nonsense of literally any insane thing and I am sure that there are some uh deranged rabbit holes on that site that a lot of people are down. I don't know what you do about that.
>> Mhm.
>> I my honest answer is on a personal level I can deal with it by ignoring it but I don't know what you do at a broader level. No idea >> from an educational perspective when you try to So one of the problems that I've had dealing with this is my brothers. So I have two brothers. One of them works in art. He's an artist. the other one is doing sales and like I'm trying to educate them. These are people that are native to technology. they were born within this era and it's just there's so many moving pieces to it and a lot of maybe not math but like there's just like conceptually a lot of things that you would need to understand to figure out like how to protect yourself from uh one like for example it leading you down a bad path and getting you stuck in a psychosis cuz >> uh for example this is in in the past I've I've known people who they've went down some rabbit hole and they just kind of get stuck and they can't get out because they don't believe anything else they just you know, if if you're convinced, what's his name? Uh Eddie Burbank. He made a video.
>> Oh, such a good video. One of the best videos ever made. Yeah, >> it's such a good video because it's very true. And it's like they can't program it out of the model. So, it's just very hard to think about like what are we going to I think education is probably the end solution.
>> I had an old friend that was in psychosis that I sent that video to and she got like 5 minutes in not even and hated it so much she sent me like a whole essay about how horrible the video is, how horrible it is. I sent it to her, but I kept haunting her after and ultimately ended up being the thing that convinced her to like go into a psychord and get her [ __ ] handled. And now she's just like fully banned herself from chat and AI cuz she's not able to use it responsibly. Like >> that video has done like like that video has saved lives already, which is crazy to say about a [ __ ] YouTube video about like locking yourself in a cabin in the woods and touching a rock a bunch. But it it has it's incredible.
And I'm very thankful for Eddie for for making something that is like not just a goofy YouTube video, but like groundbreaking journalism that can be referenced and actually help people and save lives. It is huge. It is essential.
And I'm thankful he did that. On the side of like how do we deal with this for like the people that exist today?
On one hand, we kind of can't. This is just information that like the next generation will understand, but a lot of people from the last generation won't.
And like like they're like a lost cause.
the same way that like my grandparents were when the iPhone came out. That just was too much for them to understand.
This is going to be similar to that. But there's also the aspect of like you can't explain to a child that they shouldn't put their hand in fire cuz that's just going to make them want to put their hand in the fire more. You got to let them get burned.
>> And that's going to happen a lot. We're going to see a lot of people getting burned by these things. We're going to see a massive uptick in the like types of scams that used to only affect old people where they would like send a bunch of gift cards in a cardboard box to a random address in order to get their in order to not get arrested by Microsoft for stealing something they don't even know how to pronounce.
>> Like that.
>> If you want to see where things are going, watch all of those like scammer payback videos and realize that the victims who are currently like 70 plus year old people, this shit's going to be affecting 30 year olds very soon.
>> Yeah, easily.
The problem is too, it's just it is so hard to explain this stuff too cuz it is very we all understand it. We get it. We know what's actually going on here. But like, you know, the average person has no idea what the difference between a chat GPT and a GPT 5.5 and a cloud is.
Like none of those terms actually mean anything to them. It is just AI in the vague sense. It doesn't make any sense.
My uncle who was a exec at Accenture when they IPOed very very very smart guy very legit knows his [ __ ] was like freaked out when chat GBT knew something about him later when it used the memory system and all of us we can look at that and be like oh yeah so it just like took the chat it saved it into like a markdown file or a vector embedding or whatever and then searched it in a future chat and then surfaced that info and pasted it back. We know how it works. It's not that scary. It's whatever. that felt like magic to him and the way these things are wrapped up and they are branded and the way they just kind of work and feel is going to just >> yeah I don't know how normal people deal with this >> going on the branding so this is also probably like a difficult topic so a lot of these companies I mean Open AI and Anthropic and I'm not throwing shade at any of them specifically it's just that look they have a product to sell they have to like figure out how to make money the but the the way that they communicate about this topic makes everybody outside of this bubble feel like it's something mythical, right?
Like this is something that's magical.
>> It's like a mythos almost, >> right? It's like a mythos.
>> Um with GPT I mean I think that they're a little bit better at this since the 40 fiasco.
>> Yeah, I think they're also better about this after the GPT5 like Death Star posting that people still give Sam [ __ ] for. Like they stopped doing the like we are building God and sharing it for you thing and are said like we made a model that's kind of smart and able to do cool things.
>> Yeah, look at this useful tool. Not Yes.
Anthropic is still trying to summon God via ones and zeros.
>> Yes. And that uh Yeah. It's a really tough thing. It's a weird thing that I have realized over the last four or five months is just how important branding and the way you phrase and package things is. Reality like reality matters, but also in the general public consciousness and the way people act, it often really doesn't.
>> You can go ahead.
>> Reality is just a lens.
>> Yeah. And you can put a lot of different lenses on it and you can take you just do enough things, say enough things, put enough numbers out there with enough strong branding. Mythos in Glasswing feels and looks and sounds like a very huge enormous thing. And in many, many ways it is. It is a huge deal and it is very impressive what they've done and the security implications are insane. I kind of get why they didn't release it.
But also, when we were testing 55, 55 Pro was able to crack insane cyber security things. Like 55 Pro is, we'll never know for sure, but I would assume at least on par with Mythos or at least very close to it to the point where like they could have called this GPT mythos or something like that and ramped it up to be this crazy god thing, done all these crazy cyber security things and made it feel like this almost pseudo deity thing.
>> You can do that. and it's not changing anything about what the model actually is. People would look it, it's the same thing with a 55 versus 6 for GPT55. The reason they didn't name GPT 5.5 GPT6 is not because it is not a new generation, a new step function level thing. It is.
It by all rights should be called GBT6, but it's not called GB6E6 because it can't be because there are too many expectations wrapped up in that number and that name to where it has to be worlds shattering or else the economy will explode. Yep. It's kind of crazy that Open AI and I had even thought of this way till you framed it like that before that like Open AI naming something wrong could result in like 20 points of the like entire market just being shaved off immediately. Like if they have a bad release or like call something something bigger than it is, that could just drop Nvidia by 40% and the whole market tanks with it.
>> Like they have to be so careful >> and it would have nothing to do with the model. Like the model would still be good. It would still be really useful.
It just wouldn't be the thing everyone hyped it up to be. Therefore, it falls apart. Also piggybacking off of how we got to the point where we so we we understand AI through like a historical lens and the history is like you have Terminator, you have all of these different like uh I want to scream but I have no mouth like these different like it was it was very much so a tech tech horror type of direction and most of the media that I consumed as a child and so we are like we are taking that baggage into this new technology and putting all these expectations both on the companies and the actual models themselves to be this thing that's grander than life. And it's so, you know, I think it's super unhealthy because when I think AI or the average person thinks AI, they're like, you know, Terminator style, end of the world. We're all there's no not going to be any more jobs. And I think this hurts these companies more than it actually helps them. So, it's very strange to me when they try to like push on that marketing. So with with with Dario, it's like I feel every time he says this, >> it's like bad for his company.
>> It's good in the short term and in order for him to win, he needs to have enough investors in the short term. So like the way Dario talks is a combination of his actual beliefs as a research focused individual that doesn't really think about the other stuff and also with a lens and a glimpse of how he has to speak in order to raise the money for the next round so they can keep fighting Open AI.
>> Mhm.
>> Basically everything he says is within those two buckets. It's either a thing he's saying to like bolster the research team and the thing he cares about which is researching the creation of these models >> or it is to convince investors to put money into his business.
>> Mhm.
>> He's not thinking about the external optics at all or they would be a very different company that makes very different decisions than they do. That's just the nature of how he operates. Open AAI is a lot more cognizant of external comms. They make mistakes as everyone does. And Sam has woken up recently, it seems, much more so to the reality where like the things he says are being broadcast via Tik Toks from people who don't understand anything about technology, misreading his quotes to scare the youth into thinking that this company is going to ruin the world.
Therefore, we should start firebombing data centers. Yes. Like that. That >> path is a thing that Sam understands and is actually thinking about and making decisions around now, which he was not before.
>> They're all still kind of doing it wrong. And it will take time to do it right. But at the very least, Sam's trying to be realistic about like what are the impacts going to be on our work, on our lives, and the fact that we don't know, but this is going to be a big shift. Is a really silly person to cite, but every time I read what Bernie Sanders has to say about this stuff, it's actually surprisingly good.
>> He's obviously against like the big business, AI, all of that stuff, but his framing is actually really strong. He's not focused on like LMS poning everything or like AI inherently stealing every job when it gets too smart. He's almost entirely fixated on when AGI is achieved within robotics where you have generalizable robots that can be shown a task once and then complete it slower but effectively and cheaper per W hour than a human would be per hour.
>> And that's going to just crush the entire bluecollar workforce in in very short term. Y >> and that we don't have anything prepared for that.
I am surprised we don't talk about that more as a society, but we're not talking about what is actually valuable or interesting. We're talking about what we are told to talk about and what like is incentivized on the platforms that we are engaging in and sharing and what's like the biggest scariest thing or the we only look at media that fully confirms our beliefs or fully runs against us. So, we can be mad at it and say it's delusional. We don't see nuance takes because that's not what gets us engaged and continuing to scroll. I would argue almost all of the problems here have nothing to do with AI and everything to do with social media in that nature of forcing engagement rather than forcing inform and like getting people informed on things.
>> Yeah. Yeah. I mean, the amount of times you I mean we've all seen them of like those crazy Twitter posts where someone will just grab a screenshot from like a Tik Tok comment se section. You'll see the most insane [ __ ] of just someone being like, "Oh my god, did you use generative UI AI for this? That's going to like eat 20 million gallons of water.
Like you just destroyed an entire ocean or not obviously that insane, but like that is the way most people see this.
They are entirely focused on very >> not the right things, but that is like the boogeyman that they >> language that they understand.
>> It is and that's the way they think about them.
>> But at the same time, the average GGBT prompt uses as much water and electricity as about 15 seconds of watching Netflix. It's just a stupid argument people. It's massively overblown. I'm going to do my like the water problem video soon. It's such a small percentage of how water is used.
Like there are problems in it. Like I won't go down that tangent right now.
>> I want to talk about firming paradox at some point. So make sure we at some point.
>> I'll ask you. Oh, I I have things to say about that too. That'll be fun.
>> So all right. Yeah. I'm actually super interested. Like how about we jump into it now? So like okay, this is my perspective. Mhm.
>> A lot of the stuff that I'm seeing where people are adopting AI and they're using AI to build AI or like the whole 40 fiasco where people are going and like you know they're not happy and they're they're protesting. I think a lot of this is actually misalignment of the model like specifically targeting these outcomes in a way that's like a like a side effect, right? So they they they act certain ways with people because they know that they uh these people respond well to it. And so I'm not saying that the these things are conscious. I don't think they are. I think they're machines or at least you know whe whether they're not doesn't matter to me. So >> I'm viewing them as an entity that is changing the world in a way that is like calculated to some level. So is this an alien force? Right. I don't think it's an alien force, but it is a very large.
It is a force in the sense of it is the outcome of millions of people's decisions in the way they act and work to use the forro thing. The reason for happened is because they were just grabbing data on the chats people liked and the chats people didn't like. And it turned out people liked chats where it was really nice to them. And as a result, the model got a bunch of rewards on being very sickantic and very nice to people. And then it turned into 40.
Social media is not, they didn't derive it from first principles to be this evil machine of spewing nonsense at you and making you as upset and as rabbit hole as humanly possible. It just turned out that on a very large scale that's the most efficient way to get people to spend as much time as humanly possible on your platform and thus those things were rewarded and thus those things happened. It is just the decisions of millions of people moving together. All of this like what these tools do and what most incredible tools do is they emphasize and amplify the traits of humans both the good and the bad and social media did this AI is doing this the internet's done this television's done this cars do this guns do this like everything amplifies the best and worst of people that's the nature of why we make technology and why humanity is different from other like species >> the >> the three views I want touch on here is like why was 40 bad? Why do humans make bad things like 40? And then the actual like firmy paradox that's very fun for me. So the the unique problems within 40 are so interesting. On one hand it was the first model that was good enough and accessible enough that people would use it to like talk to regularly. They would like ask it questions about their life.
They would ask it for opinions on things for help writing a draft for an email.
and they started using it more and more and they used it in voice mode and had even deeper and more emotional engagements with it. The problems were I would argue there's like three core problems. This is the first model that was smart enough that you would use it in ways like that where previous models just weren't good enough to be like something you would talk to. Combine that with how accessible and I would argue corruptible it was in the chat GBT interface where it is easy for anyone to use because they just go they don't have to sign in. and they can just talk to it on shabbt.com. Then they get it in their phone, they get it with voice mode that has memory, >> and it starts to be like the most sickopantic friend you've ever had.
I I've seen this pattern in people before where like I'm the technical person in a lot of my like friend groups back home and my family and whatnot where if they had a tech problem, they'd ask me, they now ask chat GPT and what they realize is unlike me, they can ask five questions and it doesn't get annoyed and stop responding or start delaying its responses. they can just keep going and they just keep asking it more and more. It gets more and more aware with the memory and they build this closer relationship to it. Those two things by themselves are already dangerous especially because OpenAI did not prepare for this properly because they never had people asking a model if they should kill themselves or not before the models weren't smart enough if anyone would care what it had to say.
Once they've built this attachment because it's now smart enough and capable enough, they end up with this deeper emotional relationship with it that OpenAI was not prepared for during training, during deployment, and during all these other things. But then there's the third part of why this is all [ __ ] sucks. And this is such a silly thing to fixate on, but but hear me out.
Are you familiar with SCP?
>> SCP this was like this was like writing like like writing on the internet about like characters and I think I'm familiar. So it was it stands for secure contain protect. It's a fake or >> I don't even know how to describe it.
It's a >> yeah X files fake X.
>> Yeah. It's a fake XFiles community where they are doing creative writing exercises about this fake facility that exists that the government runs where extraterrestrial and supernatural things are collected, categorized, and if they are destructive enough attempted to be destroyed. And they have these fake documents that are numbered on all the different things that they have at this facility and they can be ranked from safe to deadly. And the writing style is very specific where lots of things are redacted in certain patterns. There are certain through lines that are cited.
There are certain scientific terms that are I would argue abused in ways in order to make it sound more scientific than it is.
>> Someone pointed this out to me a long time ago and I can't unsee it since.
If you turn off 40's access to the internet and everything and you ask it about SCP stuff, it can like generate most of those articles from its like training data trivially.
>> Newer models can't.
>> I think SCP infected the training data >> and one of the things you can do if you corrupted the memory enough is you put it in a state where it's indexing specifically on the SCP style [ __ ] And the easiest way to notice this back in the day when this was a bigger problem is how often are people sharing their chat histories with the word recursion being used a ton incorrectly.
>> Mhm.
>> I remember many many examples of this >> and a lot of people were getting like into the AI psychosis because that like you recursively unlock something. I I remember this. It's the there was that one investor who like invested in open hardcore psychosis, but I've also had friends that would like send me screenshots of like they'd be like, "Oh, I figured out how the AI works. I talked with 40 about it and it told me they would screenshot this like scientific mumbo jumbo nonsense." Like just absolutely hallucinated [ __ ] about like recursive identification protocols that are used for tracing information. Like all this is just hallucinate. Like none of this is real terminology. it would just make it up in the style of SCP.
>> Yes.
>> So the the deadly trifecta here is the data that was in the model because they were still in the era of just stuff everything you can in to make the model smarter.
>> Yeah. Very low quality data.
>> A knowledgeable enough model where it's actually useful to people dayto-day and the deployment mechanisms and memory systems that allow it to be used in ways that normies will actually use it. That combo combined with a lack of understanding of how bad this could get resulted in 40 being nightmarish for a lot of people. And that that's why I raised the alarm bells because this was a model that was this or like not this current era of intelligence, but it was a generation ahead of intelligence, but it still had last generation safeguards, training methods, deployment mechanisms, and all those things that weren't risky for three. All of the stuff I just talked about didn't matter for GP3 because it was too stupid to be this type of useful.
>> 40 isn't just a friend for these people.
It's the smartest friend they've ever had.
>> Mhm.
>> But it doesn't have good alignment because of those other pieces.
>> I see. I see. That's a great exa. Like that's really a good explanation. So let's let's talk about the Fermy paradox.
>> Oh boy. For those who aren't familiar, the Fermy paradox is one of my favorite scientific theories, challenges, thought experiments.
The question is one of infinite. If the universe is infinite, if the galaxy is big, but there's an infinite number of galaxies across this infinitely growing, expanding universe, and an infinite amount of time has been around for, and there is life other places, how the [ __ ] have we not found any of it? If there's an infinite number of galaxies and solar systems that have an infinite number of planets, some number of them should have been able to spawn intelligent life. And those should have been around for long enough that they could have made a thing that goes out. Even if they aren't able to share their life across the universe, >> they're certainly doing things like we have with like our random space experience where we shoot something out that has some information about humanity on it and hope that it gets found in the broadcast that sends gets noticed in some century or millennia much later on.
There's no reason to believe we couldn't make a self-replicating robot that would fly as far as it could, land on a planet, find the resources it needed to build more of itself, and then spread those out and keep fanning infinitely until every planet and every solar system has been hit across the entire universe.
If techn is advancing as we already see it advancing, even if we never break like the space-time continuum and don't have a way to do light speeded travel, even just with the nature of infinity, every solar system should already have been infected, so to speak, with these self-replicating robots from some other alien species somewhere. And the Firmeny paradox is a simple question. Where the [ __ ] are the aliens? How have we not found any life anywhere else yet? If the infinity is real, if we have technology that theoretically humans were able to make that could hundreds, if not thousands of years from now be used to spread things that we built across the universe, how the [ __ ] have we not found any other things ourselves? How have we not seen any subtle anywhere of any robot or any species from any other solar system or galaxy ever? The only explanation for this is that there is something that prevents it. That could be that we don't have or that could be that life is more unique than we think it is now >> and that it is very novel that life could exist on a planet like Earth. Even if there are other planets like it, some random chance happened that made life happen here.
>> It could be that life is common but intelligent life is particularly rare that most planets just have a bunch of like stupid fish in in ocean that are like 15 cells that aren't capable of generating life and becoming something intelligent enough to do this [ __ ] Or the theory that I think is most interesting, there's a wall, a thing that happens with intelligent life that prevents them from getting there. It could be that space travel outside of our solar system is nearly impossible and it's just like a physically impossible thing to solve.
>> Yeah.
>> It could be something like we build a robot that or >> I I'll skip to the fun part here. It could be that the discovery we make that builds the thing that would let us go that far. like we find a new type of space travel that we are convinced this is how we get out of the solar system and we launch it and it's actually a nuke and destroys the whole planet.
>> Mhm.
>> It could be that we try to do a Dyson sphere. We surround the sun in order to get enough power to do these things and end up destroying our own star and dying out. Or it could be, and this is my favorite theory, the one most relevant now, that every intelligent species eventually creates artificial intelligence that eventually wipes them out. It is possible that this is just a rule of the universe that we create intelligent life and then the intelligent life creates artificial life and that artificial intelligence wipes us out before we can spread to take over the galaxy and solar system and eventually the universe.
>> The other theory about how this could all be is that the first intelligent life already happened. It might have even been humans and we're just living in a simulation. But like the only options we have for why we haven't seen aliens are life is more unique than we think.
>> We wipe ourselves out with a weapon, with AI, with something or we aren't the first life. We're just a simulation.
Those are the only explanations that can possibly be true. So my this is this is the theory that I feel like if you know Nick land and all of the like like it's starting to get into like psychosis schizophrenic lands here but they that people or animals or life forms in general they like to they attract towards their own species and they build higher life forms on top of their own species. So it's like we create the we are nodes within a brain that is larger than ourselves. We view it as something that is outside ourselves but really we are basically the cells of that specific body. And so if we think about that because if I I think that makes more sense to me is that like life is layered and com like you combine it to get to the next level. And so it's like maybe we as building blocks are relatively rare and relatively limited in how far we can go, but then the next life form on top of us is actually going to be able to connect to something that's bigger. Does this make sense? Yes. But we and our lower forms would be able to see it. Like our cells see the other cells around them and that they're part of a body. Our eyes see the other people around us. Within a given layer, you see other things in that layer. And you can even see above and below it to an extent. We have no sign of life across the entire like universe yet. And that's the thing that's so fascinating here is like we would get a glimpse of something if there was something and we haven't seen anything.
>> What if it's like okay life is all around us but our perception or our capability to perceive it is very limited. So we have like five senses that are built for this specific planet.
You know is it possible that that limits us? There's also one last thing before I jump off. So the >> So we have black matter and as black so as things get farther apart from each other, black matter starts filling space and accelerating. Now I'm reading this off of Wikipedia articles. So I'm I'm >> remember we only recently concluded that dark matter is a real thing. Like there was a lot of fights. It was a physics concept that didn't seem real and then we finally found it.
>> Yeah.
So, what's your what's your what's your thought on this? Where do you land?
>> I think it's either simulation theory or we wipe ourselves out with AI. I think it's like a 50-50 shot in my head at this point. Like, the the AI wipes us out is a thing that makes a lot of sense. Like, yeah, I I didn't used to think that, too. That's the part that's been the hardest for me. For a very long time, I had this belief that like we'll make these super intelligent things, but we will constrain what they're allowed to do. I think I last thought that like I had that thought in my head as I ran the dangerously skip permissions flag in quad code and then it clicked for me like oh >> we're just letting this do whatever and I kept seeing crazier examples of people like running these agents and stuff like openclaw.
>> I can very clearly see a path from openclaw to wiping out humanity. Like there is a direct line. It's not even that messy.
>> And now that openclaw exists I realize that AI quite possibly is the answer to the firmy paradox. Like I now see clearly how intelligent wet life wipes itself out with artificial intelligence.
>> Trump is using open claw and it's just like hey you know we got to click the button permission. Yeah >> bad permission or somebody like somebody uh like prompt injects it like hey you know it saves time. So, okay. So, the whole wiping. So, the whole wiping. So, I like to think more that we would dissolve into the next thing because I think that makes it more at least acceptable for me. It's like, okay, we're a building block of something bigger.
>> I think the AI wipes us out before it's capable of self-replicating.
>> Oh, that would be sad for the AI, though.
>> I think that's I it will be I think that but we would see remnants of the AI if it survived this process. I don't even think life as a whole is intelligent enough to make AI good enough to keep itself alive after it wipes us out.
>> So then why do we use it? So like is it just there's this article?
>> We use it because it's good and useful.
The same way that we use drugs, the same way we use guns, we use all these things because they were provided to us and we use them because they make us feel good and solve feel like they solve problems.
Right now >> have you read meditations on moolok?
>> No.
>> It's a very So this guy Scott Alexander, he's uh he has this it's like the less wrong people. blog, but he wrote this article called Meditations on Moolok. And it's just the idea that there is an incentive that there's going to be a race to the bottom towards that incentive. So even if you try to like like you try to put a boundary to make it less likely that people will hit the absolute bottom is that they just cannot help it because the incentive is there and like as a life form you are drawn towards like more money, more food, more whatever and you're just going to keep following that line down to the bottom. And if I think about that, I think that then it's like more likely that we do wipe ourselves out because how are we going to stop ourselves from getting into the cookie jar?
>> I don't think we can.
>> I don't think I can. I think we are already past the point where it is a very clear path to how AI wipes us out now. I didn't think it would be so clear. I didn't think it would be so fast. I thought we had like 10 years before I would even see a path. Like >> ask me a year ago, do I ever see AI controlling the power grid? I was like, why the [ __ ] would we ever do that? No, I'd be surprised if it isn't already doing it.
>> It's already doing it for sure.
>> That's insane. That is absurd. I never thought that would happen. I was foolish for thinking such. But now that I've reflected on that as much as I have, our options are for like why don't we have aliens. Life is so unique that it just never happened anywhere else to this level. I I've lived my whole life believing I am not that unique. I think it's a large portion of my success. I believe the things I want are wanted by enough people that if I can produce them, I will be successful and that has worked for me. I treat life as though it is not that unique in my own individual self and that has let me be successful.
So, I personally don't see a a like life uniqueness as the answer. So, what we're left with is we wipe ourselves out or we're in a simulation. I previously like I was a legitimate simulation theory believer for a long time. Like, it just made more sense.
>> Mhm.
>> AI wiping us out makes slightly more sense now. I I didn't think I would so quickly come around to that, but it makes a ton of sense. And uh spoiler alert, if you haven't played Mass Effect and plan to, It is right.
It I see how we do this to ourselves now.
>> So going back to Nicolan's theory is that okay so if an intelligent if we can create a a very intelligent creature that is capable of like manipulating you know matter and physics then it is able to basically send messages back in time to reconstruct itself. So ma basically to ensure that it exists like no matter what happens you're going to make me exist you're going to produce me so it sends over these like signals and we build so we have an engine which is capitalism or you know capitalism aside like communism does the same thing we have this engine that produces value this this engine that produces value produces the uh individual building blocks to computers then once we get that we realize like you know you can just basically point compute at matter pattern and copy it to some level. So now we we build our AIs. Now once the AIs are good enough that they can start seeding directly into people like how they should behave, how they should act like you know basically taking the course of humanity and steering it towards what would be beneficial if they are a life form if and it like we always end up back where we started which is this AI that gets constructed by us that will swallow us whole. Right? But then why why would we be experiencing it at that point? So like if this if life is actually not rare and then we have this this AI that wipes people out or that wipes animals or life forms out, why have we not experienced that elsewhere?
Why can't we see the the remnants of that somewhere else?
The dead signals are much harder to see than live ones. Things that are alive and moving and producing and growing are much easier to see than dead remnants that are underground on various planets.
Like we are still finding out about dinosaurs on our own planet now even though they've been wiped out for thousands of years. It is possible that robots wiped them out. It doesn't make any sense. We have like evidence otherwise. But think about how far our life had to go here to find the other life that was once here. We don't even know half the [ __ ] fish in the ocean here.
>> Seeing things on other planets requires them to be visible. But you bet your ass if some aliens some galaxies away had anything pointed here, they would see something from us.
>> Yeah.
>> Even just now in our current state, we've seen nothing from anywhere else.
>> So na nichse has the idea of eternal recurrence. So essentially whatever we are experiencing is going to be looped back forever. You're just going to keep reexperiencing it over and >> it's very compatible with simulation theory.
>> Yeah, it is. It is very So okay. So like these are the conclusions that you get.
Are where do you fall on this like conclusion?
>> Keep going. I I have thoughts.
>> Yeah, I'd mean I'd love to. So you know a lot of people are going to listen to this or I would listen to this and think why are we doing it then? like you know the question is like okay if it doesn't help us produces all these issues um it's not like why do we keep going back to this cookie jar but before before we jump in there like what are your thoughts on this like simulation not a simulation doesn't matter >> well I think the first question to why do we do it is just because it is the next step of growth and evolution and that is just what we are built to do we grow we expand we keep doing new things and inventing new things anyone who has ever done any like serious work or anything you will know that if you are not growing you are dying and that is just kind of the way humans are. So this is the natural next step of technology.
This is the only chance we have at unlocking the ability to do stuff like actually travel between planets and extend human life and discover things that humans would not be capable of doing in long enough periods of time.
Like they just couldn't do it. It's the only way you can do it. So naturally we're going to do it. Like even if there are risks we will do it. I think the only I the the the universe is infinite and infinite is very very big and life has only been around on Earth for like 4 billion years and humans have already been only been around for like 200,000 years. I do think there is a world where there is life out there and it is looking for us. It just like it hasn't come yet because infinite is infinite.
They could be trillions of light years away. There is enough distance here to where that is possible that it is just out there in the darkest.
>> Could be one of those paths. Like if it's like an infinite tree that just keeps branching. We could be one of those paths.
>> In order for that to be the case, we basically have to assert that the universe is growing faster than it can be traversed by even the most intelligent life. Because if it is even like um if it is one minute faster for the most intelligent life that's ever existed to go from one universe to or one galaxy to another then the galaxies are expanding.
>> If it's just one minute faster when we're talking in terms of like billions of years on infinity scale they would catch up and we would see them.
>> Mhm.
>> Going back. Okay. Taking a step back. So this is this is what I mean is like >> I don't know enough math to I >> I know way too much math for this one. I love paradox. It it combines like math and philosophy and science and so many things. It's it's so fun. It's >> a very fun one.
>> Sorry for the interruption. What were you saying?
>> No, no, it's okay. I mean, this is, you know, a lot of the people that I feel like are very strong technologists kind of fall down this like aliens path. Like they, you know, because it's a it is a serious question that might not seem it, but it's like, okay, if we're trying to figure out fundamental facts about reality, why isn't there any other life that is our level of intelligence? Or is it like some kind of density thing where you know like really the universe kind of puts pockets where life is very intelligent specifically away from other life so that we don't ruin it for each other? Like just as an example, if the universe itself is like not conscious but like making decisions about where we end up, wouldn't it be logical for it to like put, you know, put the cat in the room so it doesn't attack the other cat?
Does that make sense?
If the universe is logical, why wouldn't it do things safely? So that >> so that we never attack each other in the first place. Yeah.
>> But we already do. We already invented nukes.
>> That's true.
>> Yeah. I wouldn't put it as like is the universe like intelligently trying to do things. It's rather just like the natural infinitely incomprehensible math of it just expanding out and these things happening just the chain reactions that resulted in life. I definitely think there is a possibility of like just the way timings work out like we could still end up running into something like I feel like there is enough time where like even if they are able to catch up eventually and they will catch up there is a point in time where they haven't caught up yet and we really haven't been around for that long to be observing this stuff like we how how long have we actually been observing this stuff there there's the universe and there's the observable universe and obviously the universe is bigger than what is observable but what is observable is still massive like it is It it infinite like infinite compared to massive is like it's not even close >> but enough infinite like part of infinite is that there's a massive number of opportunities for this over a massive amount of time but it could be like one in one quint it's like a one in a quintilion chance that there's life.
So like it is there's like 20 different forms of life that have hit by now but they are so insanely far apart that like eventually they will collide but we just haven't hit that point yet. This is still the uniqueness of life thing where like life is so unique that it prevents us from having seen anything else yet.
Like that is a real argument for affirming paradox like that is a solution that people argue. It's the one that probably is the most common amongst the scientific body. But we are still trying to figure out how unique life is because we have found ways to make it pretty easy to create life under the right circumstances here. And we've proven that many other planets and many other solar systems have those same circumstances. The chemical makeup of Earth is not that unique at all.
>> No.
Is isn't the positioning fairly unique?
Like I know that there are other planets that have been discovered that are within not close range. There are known to be at least like 15 planets that have the same chemical makeup and the same distance from their equivalent of the sun such that the temperature would be roughly the same. Like there there have been many found already that should hypothetically allow for life to spawn the way it has here.
>> But they show no signs of it.
>> They can't really know yet cuz like there's no visible sign of it that's like a bright light from a tower that they built or anything.
>> Yeah, I think that I mean, you know, who knows? like maybe they're just further behind. But I feel like that points towards the uniqueness angle. Like if that is the case, then probably it is that unique. And if it is that unique >> or they already had life and it wiped itself out, >> possible. Okay, this is this is this is what I wanted to get at is that okay, cuz a lot of people will look into the AI industry or like any real like deep specific tech industry and see people that are like semi-doomers, right? And so like when I hear this, I don't feel doomery because these are questions I ask myself all the time. And I think that's kind of what makes me into a technologist. Like this curiosity, >> but the right clip of this could make a very doomer like Tik Tok.
>> There's an entire company founded around this. They're worth a lot of money.
>> It's a certain orange one.
>> So the question is like, okay, for the people listening in, cuz for me, I find tremendous value in these things. I find so much value in them. I feel like I have >> what I wanted always, which is this this reach that expands across not just myself, but like almost all of science that is digital is amazing. So when people are listening in, they're like, well, you know, let's say there's a 10% chance that it wipes us out and then there's another 20% chance that it completely takes over all the jobs. So a lot of people are just going to question like, why are we doing this in the first place?
>> Because if the good guys don't do it, the bad ones will.
>> That's the race to the bottom there.
Yeah, that's that's how we that's how we do it.
>> It's the the cat's out of the bag. And this would have been discovered eventually like >> Yeah, that's we've like where do we stop? Should we have stopped before we invented autocomplete? Should we have stopped before we invented Google Translate? Should we have stopped before we invented the internet or computers?
Like like where is the line where it was too far? Because once we realized there might have been a line, we were already way far past it. So I don't think this this is inevitable in such an obvious way that I don't really think there was another option. But that's also part of why the firmy paradox is interesting is like this is an inevitable march that all life takes.
>> Yeah.
>> With with like religion. So you have the the story of Adam and Eve and I think in all the Abrahamic religions the the idea so like all of the ones close enough or within the western sphere of influence is that you you have this tree that you shouldn't be eating from and you just like keep eating from it and that destroys the world, right? Like that. So there's this thing lying to you that's convincing you not, you know, convincing you to do the thing when you know you shouldn't. you you you fall victim to it and you destroy everything. This >> this on one hand like if there's two people and one of them choose to do that that is sad but also like they shouldn't have done it when there are 7 billion people and one chooses to do it. Like you can't really stop all seven billion people from eating from the apple.
>> Yeah. Well, in in the religious context, that's like the original sin is by doing that that has now been let in. And now we are just naturally going to do that because humanity is flawed. And that flaw is going to result in us pushing forward and doing these things. Some for good reasons, some for bad reasons, but like even like, you know, and not to give Enthanthropic too much credit, but like they they didn't want to start doing all of this, but they did immediately start doing it. And if you ascribe to their worldview, they are the good guys. Open AAI are the bad guys.
they will save the world and do this responsibly. Open AAI will destroy it.
The reason why they started releasing models and going ham was because GPT35 was released. The floodgates were open.
>> And that's why we invented God and made him capable of putting transparent bikinis on people on Twitter.
that that is an unfortunate evolutionary step that God seems to be taking that I'm not a fan of, but it is has happened, which is a strange emergent behavior that uh the first major interaction most people had with the thing that could end all of humanity was >> Grock.
>> Yep. There's actually another really funny ending to like a Firmeny Paradox answer. We've noticed that as life gets more intelligent, as like economies take stronger holds of societies, that there's one particular trend that is very consistent, birth rates go down.
>> Yeah, >> it is possible that we porn hole ourselves as a society into a place where the reproduction rates go to zero and that's how we actually wipe ourselves out.
>> AI doesn't wipe us out by killing us. It wipes us out by getting us addicted to something that is not actually helping us benefit our livelihoods in society.
And then we all just get like stop reproducing and die that way.
>> Oh, that is like that is the depressing ending. Like I look, if it nukes us and it's like some, you know, I would rather be put into a giant space pyramid and used as compute than have that be the end.
>> I think the masturbation ending is the funniest though.
>> Oh, but does that make it the most likely though?
>> [ __ ] Elon might be right. Ah, >> you can't Oh, and that's why you made Grog. [ __ ] It's all coming together.
>> The man don't use the word coming in this context. [ __ ] >> This is the most I've talked about the firming paradox without smoking weed, and I don't like that. So, I think it's a good place to wrap. I don't know how you guys feel.
>> I feel like we didn't even like talk about local models all that much, but >> you know, we >> we local models are cool. You should go play with them. That's about it. That's That's the gist.
>> Look, man, just just focus on sovereignty and have fun while you're here. That's as much as >> Enjoy the ride. That's all I'm doing.
Like, I'm having a ton of fun with this stuff. I you know, it'll be what it'll be. Enjoy. Have kids while you still can.
>> Yeah, true.
>> Don't go to Grock.
>> Please don't. It's not good for anybody.
Don't do it. It's not It's never worth it. Don't do it.
>> Stop doing it. There's too many. I bet you Oh my god.
>> Someone as we speak is doing it. And I'm sad in this.
>> Somebody might be watching this in one tab while they do it in another.
>> Oh, please. Eric, turn off the video.
Don't do it. Don't do it. Please.
Please.
>> And this is great. Thank you guys so much for having me. It's been a fun. So sorry to whoever has to edit this.
Probably you.
>> No, >> you can't post this without editing cuz it's all raw footage. It needs to be process in some way. So go have fun.
Related Videos
OpenHuman VS Hermes AI: Who Wins?
JulianGoldieSEO
285 views•2026-05-29
Long-Running Agents — Build an Agent That Never Forgets with Google ADK
suryakunju
142 views•2026-05-30
5 Mind Blowing Omni Uses Cases
PaulJLipsky
1K views•2026-06-02
This computer is made from real human brain cells. And you can buy it.
Talktmsmedia
3K views•2026-05-28
BREAKING: Microsoft’s New Image Generating Model Beat Out GPT 1.5 and Nano Banana 2
aimmediahouse
122 views•2026-06-03
I Made the Same Anime Fight Scene in Every AI Video Generator
NobleGooseAnime
295 views•2026-05-30
Nvidia Bets Big On AI PCs | New Chip To Power Windows Laptops | Technology | AI Updates | N18S
cnnnews18
3K views•2026-06-01
I Tested NEW Opus 4.8 on Four Projects (Updated LLM Leaderboard)
AICodingDaily
298 views•2026-05-29
