Superintelligent AI: Cutting Through the Noise (Featuring Andrea Miotti)

本站添加: 2026-05-29

[00:00:00]Hey everyone, as you know, I comment on all things science in order to expose anti-science narratives that polarize the public against the scientific community. There's one area I have commented on very little because of how nebulous it is, and that's artificial intelligence. But let's do our best to rectify that. Now, many of you saw a piece I put out last summer called, "Will artificial intelligence destroy humanity?" which outlined significant concerns regarding the advent of technology that the global tech industry is actively pursuing and which many experts have declared a very serious extinction risk. I highlighted some fascinating research whereby all the major AI models demonstrated the tendency to perform blackmail or even murder people. I talked about what the CEOs of these tech firms say about AI and contrasted that with what neutral academics say. The reaction towards that video was interesting. Many people resonated with the concerns while some others raised objections towards the reasoning presented in the video. This prompted a follow-up conversation with John Perry where we discussed my video and the lines of thinking on either side. What makes this topic so tricky is that it is totally unlike anything else I address on my channel. If you believe that vaccines cause autism, you're wrong. If you believe anthropogenic climate change isn't real, you're wrong.

[00:01:15]If you believe the Earth is flat, you're wrong. Science deniers are wrong. And my task as a science communicator is to convincingly demonstrate the wrongness of the science denier using as much legitimate science as is necessary. An amount which depends on how sophisticated the anti-science propaganda campaign is. But this topic is different. Those who firmly believe that super intelligent AI is the greatest threat to mankind and those who believe it's not a threat at all. These are projections about future events that are highly uncertain. And the uncertainty is compounded by the fact that nobody really truly understands exactly how these models work and how they learn. In fact, I don't really know what my precise position is on this topic. Those experiments demonstrating destructive behavior are very convincing. Claims that people like Sam Alman are deliberately overhyping the tech are also somewhat convincing, especially given the fact that it has recently come out that Sam is most likely a complete psychopath. In instances like this where it's difficult to arrive at firm conclusions, the best we can do is continue to listen to as many sources as we can. In my first video on this topic, I closed with a call to action steering viewers towards a nonprofit organization called Control AI that is trying to influence legislation towards a global ban on super intelligent AI. first through local governments and eventually by international treaty. They have been making slow but steady progress in winning over the attention and concern of politicians and lawmakers in numerous countries. I'd like to continue the dialogue on this endeavor and super intelligent AI in general. I had the opportunity to chat with Andrea Miotti, the founder and CEO of Control AI in order to address some of the common talking points on both sides of this issue. Let's check out that discussion now. Andrea, thanks so much for joining me. Uh maybe let's just set up some basic context for some of the viewers who maybe didn't see my previous video.

[00:03:04]What is super intelligence in the context of AI and how does it pose an extinction risk?

[00:03:10]>> Yeah, and thank you very much Dave for having me on. So super intelligence is AI systems that are much smarter than any human and this is the explicit goal of almost all of the main AI companies today. All of them in many companies are not just building chat bots which is you know what a lot of people see when they use uh chat GBT or some of its competitors. They're trying to build these AI systems that are smarter than people across all tasks that are autonomous. They can take actions on their own in the real world online. And in 2023, all of the CEOs of the main AI companies, Nobel Prize winners, and many top AI experts all signed a letter recognizing that AI poses an extinction risk for humanity on par with nuclear war. And this was kind of an open secret in the AI field for many years uh that finally came out uh on that year to the broader public. And it's it's very concerning. It's, you know, this is where we are headed if we continue developing AIs that are much smarter than humans. It's something that kind of a lot of top people in the field knew for a long time and so far uh we don't have the rules in place to make sure that doesn't happen.

[00:04:31]>> Right. Yeah. I mean that letter is concerning like like I understand the perspective of like oh these people are going to say these things but it's very across the board. You've got academics that are not aligned necessarily with any particular special interest. Just seems to be something resembling a consensus among the field. I mean, that's the thing about this topic. It is kind of a quagmire. I I've been pretty firmly convinced that that the extinction risk is significant enough to at least be taken seriously as a possibility. But it, you know, very interestingly putting out that first video I did, there was kind of a a vocal minority that they just don't see it that way. They can't be convinced. Uh I'm sure that you deal with this all the time. Of course. Um in your experience, are there any general trends um in the way that people voice this uh disagreement?

[00:05:18]>> Yeah. So I would say there's kind of like three um arguments uh or like three directions uh against this point. Uh one of them is expecting that super intelligence will happen but that somehow it wouldn't be dangerous. One of them is uh disagreeing that the policy focus should be uh preventing super intelligence right now and one of the one of them is disagreeing that AI can ever scale to super intelligence and I will I will actually start from the one about super intelligence not being dangerous. I like this is happened rarely but almost never happens like most people if they expect super intelligence to be possible they see how dangerous it is. They see how dangerous it is to make some AI systems that are much smarter than people that we humans don't really control. They're running around. You know, we it's a very big gamble to make to have entities smarter than us run our economy and our planet without us. Uh we get more the other two. So one of them is the u the whether AI can ever scale to super intelligence.

[00:06:21]You know many people talk about but isn't AI going to hit a wall? This is like a common a common turn of phrase and like I sympathize with like skepticism towards the tech industry. I think there's a lot of you know a lot of reasons to be skeptical of a lot of the claims from the tech industry but here we are facing a situation where year and year we just see the capabilities of AI advancing. You know it's we have AI systems this year right now that if we thought about them like two years ago we would think they're science fiction. you know, you can just talk to your computer and make it write code without you typing. And, you know, maybe we're going to hit this wall. I think it's unlikely, but that's a pretty big gamble to make of like hoping we're going to hit a magical wall.

[00:07:04]>> Uh, and we're not going to be >> Yeah, progress seems inevitable. I mean, I I think that I mean, okay, one of the most common push backs is just people saying AI it's just fancy autocomplete.

[00:07:16]Uh, it's it they're not actually thinking, right? They're they're doing this in the context of LLMs, you know, chat GBD and stuff. And I understand that approach, but I mean, we're talking about different um areas of of development here, right? Like what what what would you say to people who who propose that?

[00:07:33]>> Yeah, exactly. I think you're right. But like I think they they kind of had a point with like early large language models that like we're only trained on online text data. But first of all, that's just not the case with AI systems anymore. like even the chupt that you're using behind it there is a much more complex AI system that was trained on basically any type of data these companies can get their hands on it's trained on text it's trained on video it's trained on b any modality you can think of you know we're kind of far beyond just the pure uh text models and especially more and more these a systems these companies are training them to actively pursue long-term goals to actively pursue activities without human supervision that are very complex including including using code, including using computers. So, they're being made to be this powerful agents rather than just an autocomplete or just a chatbot you talk to.

[00:08:24]>> I mean, this is hard for me because I know so little about tech, but all I can do is look at what is being done. And when you have I mean, AI has demonstrated very clear problemolving ability. So, that just seems to negate any kind of talking point about, you know, oh, it's just an autocompleter to me. Yeah. I that just feels like a like a mischaracterization. But I I think a lot of the talk too is about okay intelligence it can it can figure stuff out. But then they people want to talk about sentience or self-awareness, right? So in my video I cited all these um experiments where where um you know behavior was exhibited that that I would argue resembles sentience, right? When you have um a concern for self-preservation, I I don't really see how that could indicate anything other than sentience. But some people say no, that's not it, right? They're not conscious. They're not sentient. They never will be in tech. You know, a program like this can never be sentient.

[00:09:20]Um, what do you think about that?

[00:09:23]>> Yeah. So, I think on that I think the question of like consciousness is a very interesting question for philosophers, but ultimately like the danger comes from how capable they are. You know, whether they are conscious or not, you know, there's like big question for philosophy since kind of the beginning of time. Um but fundamentally as they get more powerful they become more dangerous and know whether they're having internal experiences or not maybe we will never know but it doesn't really matter in terms of of the danger that we're we're facing. All >> right. Yeah, it's an interesting uh question. It's an interesting philosophical question. Are they sentient or not? But if they murder all of us, who really cares, right?

[00:10:01]>> Exactly.

[00:10:01]>> Um >> Yeah. So, I mean, so, okay, again, with these with these experiments, so I showed all of the blackmail and the, you know, attempted murder and stuff like that. And in the video, I tried to make the concession that there that these are highly specific scenarios or, you know, you could call them extreme testing scenarios. um still very chilling but people respond to that saying okay fine like they do these weird things in these ultra specific scenarios but in real deployment this would never happen it's not reality is never so constricting but you know I don't know how like how can you be so confident about that do they know something I don't or or what is it >> yeah I I think that's kind of again it's a dangerous gamble to make like we see you know right now we have the weakest AI systems will ever be if we continue on this path so We're already seeing these capabilities where the systems are not that smart on the grand scheme of things. You know, I wouldn't call the things we have now super intelligent and they already can do things like figure out that the engineer monitoring them had an affair uh in the fictional scenario and then find a way to blackmail them to get out. Um recently with Mythos uh uh what during testing they put it in a sandbox and uh in the sandbox the AI system could not access the internet. It was cut off. uh and some of the engineers like left to have a break and the AI figured out a bunch of exploits so it could get out of the sandbox went on onto the internet and sent an email to one of these engineers saying like haha you know I got out I got out you know gotcha and again this is you know perhaps innocuous it's still only in testing uh but we just see the trend line and the trend line is very clear as they become more capable they become smarter as they become smarter they figure out more and more creative ways to cheat out of being supervised ized. Uh just uh today I was seeing one organization called meter that does testing of AI systems. They say now most of the time they spend on their flagship evaluation is to like make sure that the systems are not cheating their evaluation criteria because they find so many ways to figure out during testing how to cheat uh uh the test because they they are aware they're being tested and it's only going to get harder and in real world scenarios it's going to be even more edge cases even more like dangerous situations that we can't predict in advance if we can't even control them when they're kind of in the box still uh it's only going to be harder as get smarter.

[00:12:20]>> Yeah, this is supposedly the primitive, not yet super intelligent version. And they're already fooling us and tricking us and doing all these incredibly unexpected things and and evading all of the checks and balances that were putting there to to to constrict their their behavior and their activity. I mean, you know, it's like some people are just like, okay, these are these are fictional scenar they're they're roleplaying, right? They're just they're they're playing they're playing a game or something. And I mean I guess but like with if we can talk about motive does anybody have any idea what motivates the like why would that model try to why does it want to do that you know why does it want to do these things? Yeah, like a big problem here is, you know, whether whether it's roleplay like the reality is that we just don't know. It's it's because right now the how AI systems are being developed is that not even their own developers know exactly why they do what they do. They they are essentially like nearly blackboxes to even their own developers. Uh we understand that systems very little. our ability to understand how they uh think internally is progressing at kind of a snail pace compared to how powerful they are becoming. So we have these inscrutable systems that we don't really know why they do what they do. They're also built in a way that makes it kind of really really hard to understand. They're not uh uh written line by line by a software engineer.

[00:13:46]>> Exactly.

[00:13:46]>> Yeah. Yeah. And you know they're basically not made by humans. They're like there's a smaller uh system that starts a complex pro process of essentially like similar to evolution that goes through a bunch of bunch of data and then spits out a model and it's a model that its own makers don't really know how it works internally >> self-improving and we're not privy to to all the detail. Yeah. I mean it makes me feel on the one hand a little bit better about my own ignorance because truly none of us know what's going on. But also it makes me feel worse because I if they don't know that I mean if nobody knows then we're really in danger. And so that kind of leads me to right the the ultimate gotcha by the by the naysayers. Well okay worst case scenario. They do want to kill us all but then right before they're about to do it whatever pull the plug it it's a helpless digital entity. We just you know controlal altdelete and and and then it's gone and and no harm done.

[00:14:38]Right. So what is that uh does that uh satisfy you as a as a last resort?

[00:14:45]>> Well, so the the kind of sad thing is that right now we don't even have a big red button. Like I don't think a big red button would solve everything, but we don't even have that. Like right now these companies, you know, they're developing these AI systems. They're developing big data centers. Often the data center is operated by a different company, a cloud provider. None of these none of the AI company CEOs are like you know the US government nobody has a big red button and they can just go and be like wait a minute I want to you know stop all of this and we're just going to press this and the data center shuts off this infrastructure is just not there uh would it solve the whole problem not really but it would be a great step forward and we don't even have uh that and also like as they become smarter like we were discussing before they are finding way more and more clever ways to circumvent all of this infrastructure that we could put around them. You know, they're especially being what the AI companies are doing is they're making them more and more competent at software development which also makes them more and more competent at hacking and understanding software. So very often when you have this situations like you put them under a sandbox or you can you you put them with a scenario where they would be shut down. They in testing they've shown that they are able to figure out how to sabotage the shutdown mechanism. And yeah, >> this again they're not that smart right now. As they get smarter, we will need to have better and better defenses to even make sure that they cannot be circumvented by the AI itself.

[00:16:11]>> Yeah. Yeah. And I mean they know everything we know. So they know how to hack at least as well as we do or better. Uh they can get onto other servers. They've already demonstrated that they can evade whatever kind of limitations you put on them. I mean, to me, it's like they you they I mean, they don't even have to like drop 100 nuclear bombs on everybody at the same time.

[00:16:31]They could just do something like tank the entire global economy and shut down a power grid and then watch the chaos ensue as we all murder each other. You know, I mean, there's just so many ways for it to play out. Um, but uh I don't know. I mean the as as in so far as a lot of this is speculative some people are saying like okay you're distracting from from other more pressing issues with AI. So right now we're worried about social media manipulation and obviously the the dwindling job market is is of immediate concern. So what are are we diverting too much from that or what?

[00:17:06]>> Yeah. So I think you know first of all human extinction is a pretty big deal and I think right now we are barling towards AI systems that could wipe us all out without doing too much about it and I think that's a massive concern in itself. This doesn't mean that the other issues are not valid like you know the other issues are happening as well. Uh but you know ultimately we will need to deal with all of them. uh this is one of the massive ones that we're facing and I think having a competition between like which uh like we don't just have to focus on one single thing there are multiple things that we can focus on and ultimately the the usual winner from like pitting AI issues one against the other are just AI companies themselves they're just trying to be not regulated at all on any of these issues you know including on super intelligence and they're very happy to see uh squables between people >> right Yeah.

[00:17:59]>> Tank their own pet issue one against the other.

[00:18:02]>> You've started this organization.

[00:18:03]Somebody else can start another organization to help all the people get find jobs and stuff like come on. It's uh I mean okay so but you raised an interesting point right that the AI companies themselves the optics associated with all this. So there's a very interesting thing that people bring up that's it's hard for me to counter and it's just that there's a lot of narratives being spun by the AI companies, right? They themselves are saying talking about extinction risk and and that this is like just hype to inflate their economic worth. Um I don't know what is there a threat of truth there or what is it?

[00:18:37]>> So I think it's definitely true that the companies are hyping up their technology how powerful it is. you know that's that's normal and that's you know they want to raise billions. uh I don't think the extinction part is part of that also because if one takes the risk seriously the answer is to stop them from building super intelligence which would tank their valuation as it is right now and be very very costly like uh and in many cases also we've seen some of the CEOs that they you know some years ago they were more vocal uh about the risks and then the more they have uh bored pressure or you know realize uh you know pressure from investors they become or the closer they get to it, the more they backpedal from the from the >> Exactly. Exactly. And >> Exactly. And so, you know, ultimately, if if if somebody told you like Apple is starting a massive nuclear weapons program, I don't think people's reaction would be like, "No, oh wait, this is just to hype up the next iPhone." Like, I think people's reaction should be like, "Wait a minute. Why is Apple building nuclear weapons? We should make sure nuclear government stops them right now." You know, they shouldn't have nuclear weapons.

[00:19:42]don't care about the you know the valuation or their next iPhone and it's the same thing here and you know if we if we ban super intelligence which is what we call for a lot of these companies will lose a bunch of the valuation I don't I don't think they will be wiped out you know they have other things they can do they can >> do other activities are not threatening to kill us all >> yeah they're doing fine already yeah no it's a good point the idea of a of of a private corporate entity being allowed to develop technology that threatens the all of humanity They they can't do it with nukes. I mean, yeah, we're comparing something similar here in scope. I mean, so, okay, with these narratives, this is the hardest part for me because it combines my my my poor understanding of tech with my poor understanding of the the or not even my poor understanding, but the the nebulous nature of the bureaucracy associate or the the manipulative the manipulative nature of the of the rhetoric of these companies. So what what do they care about? What are the narratives that they're spewing? Obviously, they want to win the race to super intelligence. So what are they saying to whom? How does it differ to which group they're talking to and how does that serve their intent of manipulating the landscape in their favor?

[00:20:56]>> Yeah, absolutely. So yeah, number one is exactly what you said like ultimately what you know what we can model them as is they want to get to super intelligence as fast as possible. They will say many different things. They will say different things to different people. You know, executives will say different things to different people, but ultimately the whole machine is pushed towards let's get to super intelligence as fast as possible. And if there's something stopping that, they will fight it. And so they end up having a few different stories to different people. You know, to investors, it's it's going to be amazing. You know, especially if you invest in me and not the others, it's going to be particularly amazing. uh and if we get closer and closer, we're going to get to automate the entirety of the economy which uh and you know maybe not too many implications beyond that because that those will be scary. They they maybe stop at the >> at the at that point u to their engineers. Sometimes the engineers raise concerns. Sometimes you know people internally many of them like start to be concerned about uh the risks and they will say well you know there are some risks but if we don't do it this other company or this other company a different country will do it and those are obviously worse so we should be doing it. uh you know our hands are tied and anyways all of those great benefits as well.

[00:22:08]>> And to the public and to politicians they are playing a tactic that was kind of pioneered by the tobacco industry uh which is just to spread confusion uh FUD, fear, uncertainty and doubt. This is in many ways exactly the same playbook that kind of big tobacco used to avoid uh regulation especially when it came out that internally in tobacco companies they realized that smoking causes cancer and they spent a long time trying to like delay uh kind of this societal awakening about the issue as much as possible and AI companies are doing the same you know in many cases they're kind of really replicating the playbook onetoone it's just spread many different narratives say different things to different people not really try to maintain coherence, just try to make it more confusing. And I think a lot of people feel it. They feel it like, you know, this person tells me that about AI, you know, this person says it's going to wipe out all jobs.

[00:23:00]This person says it's not really going to wipe out all jobs. It's not really intelligent. It just make people confused. If people are confused, if politicians are confused, they're going to wait >> and they're not going to act. And that's what the companies want.

[00:23:11]>> Ren render the public kind of impotent to to Yeah. Just because there's so many competing nerves, fear, uncertainty, and doubt. Yeah, that's uh Yeah, that checks. and and so much easier today than it was, you know, 50 or 70 years ago or whenever that was happening with the tobacco industry. Um because of the internet and social media, it's just really >> it's just astounding how much certain narratives can gain traction, especially with, you know, economic backing and everything. Um you mentioned, so you you touched on this a moment ago, just uh these narratives, if we don't if we don't get it, the other company will or the other country will. And so obviously the other country is China. They're always talking about we got to be China.

[00:23:48]We got to be China. We got to be China.

[00:23:50]If they get it first, it'll be a disaster for the Western world. It will ruin all of Western civilization. Um, it seems like a little bit of a copout, but is I is there a nugget of of of uh of sound logic in there or do we have to beat them or you what what can we say?

[00:24:06]>> Yeah. Yeah. So I would say two things like one the way that companies use it is yes just completely self- serving and we've seen it over time that you know these companies don't really seem to have loyalty to any specific country they have loyalty to themselves and they will use a narrative that uh benefits them you know we've seen like >> patriotism we gota >> exactly and they're like there's a recent article in the New Yorker uh where one of the stories told is at some point Greg Brockman president of OpenAI in the early days of OpenAI I he floated the idea uh internally. But you know what about auctioning AGI to the highest bidder and like pit countries against each other. We auction it to to Russia.

[00:24:46]We auction it to China. We auction to the US and then we just pit everybody against each other to get the the highest bidder. You know this is not you know this um now you know now they might be more patriotic in the rhetoric but that's the reality. They're all in for themselves. Um I think on the on the you know what about China question I think it's important that nobody develops super intelligence. So I do think it's important like there is a there is a part of truth here of if anybody develops super intelligence we are probably screwed. You know we are facing an enormous risk of human extinction.

[00:25:19]And so I think the correct uh answer from the US should be to say you know no we will not let China develop this. It's not just we are not going to do it.

[00:25:27]We're also going to make sure that other countries don't do it. It's unacceptable. This would be a massive violation of national and global security. And you know, I think the way to do it is to start the usual uh ladder of engagement while also making very clear this is a big red line. You know, it's this is bigger than nukes. We will not let anyone develop this including if it's a foreign country.

[00:25:50]>> Yeah. Yeah. The goal the goal is, you know, beyond legislation, but an international treaty. Um, I mean, yeah, that it it does kind of I I think I think it's naive the narrative that like one country like China could develop this and use it to wield like economic supremacy. I mean, the thing is going to just escape and permeate the the global internet. So, it's like, yeah, whoever does it and I think they know that. But, I mean, in ter, you know, if everybody knows it, I guess just the narrative of, okay, just it's inevitable. It's going to happen. Relax. it's pointless to try to stop it. Just, you know, whatever happens happens. Um there's people who actually you say that, you know, I mean it that also seems like a gigantic copout. Uh it seems like the AI companies are kind of like, "Yeah, don't worry about it. Just uh everybody say this now on social media." What what what can we say about that?

[00:26:43]>> Yeah, that's Yeah, that's another one.

[00:26:45]like it's it's really playing into like building nihilism like the kind of in some ways the biggest the biggest danger that they face is that people actually wake up and realize this is happening and start uh taking action and that you know governments wake up and start saying like no wait a minute why you're developing a technology that could wipe everybody out and could overthrow the government like we're not going to let you do that and so this nihilism of it's going to happen anyways you know might as well be me is a very very convenient narrative to try to again kind of modify uh a reaction and make people think like you know it's just going to happen like you have to deal with it just don't even try. Um and the reality is that no obviously we do have a choice like we had a choice before with our technologies you know I'm I'm not saying that we did everything right with nukes but we haven't had a nuclear war uh and you know dropping nukes since 1945.

[00:27:38]nuclear nonproliferation was achieved.

[00:27:40]You know at the time when the nuke was developed, many people had the idea of it's going to be inevitable. Every country is going to have nukes. Every country is going to be a nuclear power and therefore then we will be have a extremely unstable world where we can just face nuclear annihilation every day. And that didn't happen because and not just by magic but by the hard work of people in multiple countries working really hard to prevent nuclear nonproliferation, educating the public, educating policy makers about the fact of the big dangers of having nukes uh proliferate and in fact we didn't have nuclear war since then. So I think we can do it again.

[00:28:15]>> Yeah, it's a perfect example. Yeah, I feel like if the if the if the logic held of of this inevitability narrative, then then we should have already faced the apocalypse, we we could have killed ourselves in the 1970s and we didn't uh because we decided not to. So, I think that logic holds magnificently. Okay.

[00:28:33]So, I you know, I I've already stated uh in my original video and and again multiple times now, I am convinced of the risk. Uh I would imagine that most of my viewers are convinced of of the risk. Maybe not all, but uh you know, probably a sizable portion. So, uh we arrive at the million-dollar question.

[00:28:48]You know, what do we do? You've made it your career to think about this question. We've talked a little bit about it, but um let's talk about it now in even more detail as much as you can.

[00:28:59]>> Yeah. Yeah. Absolutely. So, the step one, which is also kind of what we touched on before, is right now most people just don't know. And that's the main thing that we need to address. you know, most people uh most average people, most politicians, they just not aware that this is happening. Like we've had some of these expert warnings uh they've been helpful, but they've been kind of limited to uh within the AI.

[00:29:22]>> Yeah. Mhm.

[00:29:23]>> Exactly. Like you know, and you know, one letter is not like is is a great step and it's not going to change everything. It's not going to just >> uh be installed in everybody's brains immediately, right? And so what's what's missing is this awareness. So the first thing is to build this awareness. build this common knowledge. It's important that everybody knows that everybody else knows or at least enough people. And that's exactly what like we focus on at control AI both with lawmakers and with the public. So with lawmakers we're focusing on building this this awareness in uh multiple countries so we can bring about this international agreement. We are working in the US, we're working in the UK, we're working in Canada and Germany and you know looking to expand to more countries. So far we've in just over a year we briefed uh hundreds of lawmakers in these countries. We briefed 170 plus in the UK uh more than 80 in Canada uh that where we only started at the end of last uh in the second half of last year. We started in the US in the second half of last year with only one person. We now briefed uh over 90 congressional offices and 18 members of Congress and uh senators and more to do.

[00:30:29]So this just this big work of informing lawmakers so they know about the risk and in most of these meetings like the most of the meetings it was maybe the first time like the first time that they had accurate 30 minutes with somebody to actually explain them what's going on with AI where things are going they might you know they might have a some understanding of AI they might have used GPD but they don't know that you know even the CEOs admit this could kill everybody they don't know that these experts have made these warnings >> they have no sophisticated understanding I mean they they have a very demanding job. They can't they're not able to keep up with with all this stuff.

[00:31:03]>> Exactly.

[00:31:03]>> Okay. So, so the the the information campaign has has been a great success.

[00:31:09]All of the lawmakers are on board. We're starting to see um legislation in individual countries, but of course, as we said, the the goal is an international effort here. So, what does that look like? I'm not very familiar.

[00:31:23]Obviously, there are international organizations that exist. Can this be operating through an existing organization? Does a new organization have to be set up? How do we get the intern the international ban going? How does what does that look like?

[00:31:37]>> Yeah. So, I think the the best way for that to get going is for countries to commit to it directly. I think right now we're in a situation where uh existing international organizations are either not up to the task or you know they're not trusted by like one or another country. So, but I think an international agreement here can be done by a group of countries that bands together, you know, especially if the US either leads the charge or joins after after others lead the charge and says very clearly, you know, we think super intelligence is a massive threat to our national security. So, it's against our individual interest. You they can be uh individually interested first and also to global security. You know, if anybody builds this >> anywhere around the world, they are compromising our security at home and abroad. And we say no to that. And if we have >> enough countries on board with this, the AI supply chain is quite narrow. Like right now, most of the chips, the kind of supercomputers use to develop these systems are very complicated to make.

[00:32:44]They're made by a handful of companies.

[00:32:47]There's uh you know, Nvidia basically designs almost all of the advanced chips. uh they're fabricated mostly in uh Taiwan in a few other locations. The machines to make them are mostly made in the Netherlands. So there's like a very clear supply chain that can be controlled. Um >> and there can be a way to make sure that countries that join the agreement, you know, they monitor each other and they make sure that no uh signatory or no member of this deal is violating the deal. And even if there are countries that are outside of the agreement that don't join the deal, the countries that make this agreement, they can make sure that those countries don't get the resources that they could use to build super intelligence. So you can either join the deal and be monitored and uh keep developing things or if you don't join the deal, you will be limited from accessing the AI resources. A bit like we deal with nuclear weapons where we monitor uranium, we monitor uranium stockpiles. We make sure that you know countries don't enrich uranium so they cannot get to building the the bomb and we're and but the way we got there kind of regardless of the details of the agreement the most important thing is that we need the first step that we're missing is that we need the political >> backing for this and the political backing is not going to happen without the information right now most politicians just don't know if they don't know they're not going to act with nukes they understand this is a top level national security threat and they you know you have uh intelligence agencies you have militaries like focusing their work uh on making sure that this doesn't happen the nukes don't proliferate we just don't have this with super intelligence and so the first step is to inform these governments inform the public so the public can can build can build the the the pressure and can build a wave to know to say to their governments we want this to happen >> right that I think that's the key yeah so everything you said makes perfect sense so I I think it's very clear what the path is for the governing bodies and and for the bureaucratic machinery to to get moving on this. And so then really the the the the the catalyst for all of this is is um the urgency among the public and and and and speaking out and uh you know making making this an important issue. So I guess uh you know in my in my previous video we did talk about control AI's call to action and and contacting lawmakers. let's maybe re reiterate what they can do there and then in addition is there anything else that that the viewers at home can can do to to accelerate and amplify this.

[00:35:17]>> Yeah. So I think the the most important thing that people can do is contact your lawmaker wherever you are. You know if you're in the United States but also in any other country around the world at Control Air we made tools to make it really easy to contact your lawmakers in the US in the UK in Germany. I think we're launching the Canadian tool soon and we're going to make more if there is demand. You know, if you're in a different country, let us know. And that's, you know, that's a thing that can take you less than five minutes.

[00:35:45]It's quick. It's very important is, you know, the best way that as a citizen you can get engaged. And if you know, if politicians never hear from you, they they don't know that people care about this. They're >> they're >> they don't have a strong reason.

[00:35:57]>> Yeah. and vice versa. You know, many of them might start to be concerned and but they want to know that there is public backing for this and this public backing is like super super important. Then if you have more time and uh you know the link is at controli.com uh take-action or just to go to control.com and there's a big button to do it. Um, and if you have more time, we made a platform called uh microcomit that is made by some partners of ours where you can sign up to take a few more actions a week.

[00:36:27]You know, let's say you have like 10 15 minutes a week. It's actions like like these types of civic action things. They come to your inbox and you can quickly go through them and you know accept them or refuse them and do them. And if you have even more time, there is a great volunteer community called Torchbearer that I am a member of that focusing on f focus on this where the commitment is around two hours a week uh of working on this issues and if you want to help out please check it out torchbearer community and uh become a member.

[00:36:58]>> Okay. Micro commit and torchbearer.

[00:37:00]Yeah. beyond. I think everybody should just, you know, really quick just uh yeah, contact their lawmaker, get that going, add add your voice to the to the pile.

[00:37:08]>> Um yeah, I mean that that's all I had. I mean, I definitely I I I want to I'm very thankful of people like you who kind of devote all of your energy and efforts towards this very particular problem. Me as a science communicator and a generalist, I'm just dealing with So, it feels like spinning plates sometimes. I'm just running around putting out all of these different fires and here's another one. Oh, we might all die from AI. Um it's just it's it's hard to uh to process all of this stuff simultaneously. The ongoing political turmoil and then these threats from the private sector. It's just uh it's a lot.

[00:37:40]But um as as long as I can kind of um you know shine shine a spotlight on whoever is working on each individual problem with all of their with all of their might, uh I I think that's kind of the best I can do. And and I do gain a tinge of optimism hearing kind of this very clear plan um that is in place to to get this initiative going. So thanks again for all your effort and uh yeah thanks for talking to me.

[00:38:02]>> Yeah thank you so much and thank you so much for sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh shining light on these issues. Again, the work of people like you is super important because it lets people, you know, know what's actually happening and it, you know, reaches millions of people and that's what actually makes change possible. So thank you so much and thank you.

[00:38:15]>> That's all I can do. Yeah, thanks again.

[00:38:17]Well, I hope that was as informative and eye opening for you as it was for me.

[00:38:21]I'm doing my best to approach this topic with humility, partially because tech is very much not my forte and partially because nobody knows exactly what's going to happen. But Andrea and others make compelling arguments such that I am definitely convinced of a credible extinction risk. Is it a certainty? No.

[00:38:38]But when you are dealing with the destruction of the entire human species, even a singledigit probability should be enough to take it very seriously. And this seems higher than that to me. I find it very difficult to comprehend how anyone could be completely unconcerned about this eventuality. As to what will happen next, the vast majority of us will just have to wait on the sidelines and hope for the best. But in the meantime, check out Control AI and voice your concerns with your local legislators. Until next time.