LIVE: The Existential Threat of AI and the Need for International Cooperation

Added: 2026-05-07

[00:03:35]stage our panelist for the evening and Senator Bernie Sanders.

[00:03:59]Let me thank uh everybody for coming out tonight. Let me thank our panelists uh for being here. Uh Dr. M Max Tegmark from MIT, Dr. David Krug from the University of Montreal, Dr. Zang Yi from Beijing Institute of AI Safety and Governance, and Dr. Schue Lan from Chinwa University. Uh, and I want to offer a special thanks to our Chinese guests who are uh up at 7 o'clock in the morning to be here. So, we thank you very much.

[00:04:38]Um, Most knowledgeable observers believe that we are at the beginning of the most profound technological revolution in world history. A revolution that will bring unimeaginable changes to our society in the months and years ahead.

[00:05:09]The scale, scope and speed of this transformation will be unprecedented.

[00:05:18]According to Deis Hassabus, who is the head of Google deep mind, the AI revolution will be 10 times 10 times bigger than the industrial revolution and 10 times faster. In other words, this technological revolution could have 100 times the impact of the industrial revolution.

[00:05:47]And it's not just what AI companies are saying, it's what they are doing. Over the course of this year, four major AI companies are expected to spend almost 700 billion dollars building data centers and tens of billions more on research and development.

[00:06:12]That is equivalent to what we spent on the entire Manhattan project which built the atom bomb during World War I every three weeks. That's the magnitude of what these guys are spending. We are already starting to see AI's impact.

[00:06:36]There are estimates from very credible sources that tens of millions of jobs could be lost here in the United States in the next 10 years. Whole professions wiped out and young people finding it harder and harder to land entrylevel jobs.

[00:06:59]Psychologists worry about the mental health challenges facing our young people and about the increased isolation they experience when they become more and more dependent on AI chatbots for their emotional support.

[00:07:19]Civil libertarians tell us that AI will be able to analyze every email, every text, every phone call, every website visit, every purchase that we make, and that our privacy rights may well be eviscerated.

[00:07:40]Political scientists worry that AI could threaten the integrity of our elections and political institutions where voters will find it increasingly difficult to tell the difference between truth and lies.

[00:07:56]And yet, as enormously significant as all of these profound changes might seem, there is another AI development that could have an even more frightening impact.

[00:08:12]If AI becomes smarter than human beings, as many scientists believe will happen, the human race could lose control over this technology with catastrophic consequences. In other words, the richest, most powerful people in the world are now building a runaway train with no breaks.

[00:08:38]They acknowledge that they don't understand how it works and they don't know where it is heading. And that is not just Bernie Sanders talking. Just a few days ago, right here in the US Senate, exop AI board member Helen Toner said, and I quote, "AI companies are deadly serious about building machines that will outperform humans at everything, and deadly serious that they don't know if they'll be able to control the machines they create."

[00:09:18]Yosua Benjio, the most cited living scientist in the world, says, quote, "We are playing with fire and we still don't know how to make sure the machines won't turn against us." End quote. Jeffrey Hinton, the Nobel Prize winner go and the godfather of AI says there is a quote 10 to 20% chance for AI to wipe us out. end quote. Andrew Yao, the Turing Award winner, says AI could pose existential risks to humanity. Quote, "Once large models become sufficiently intelligent, they will deceive people."

[00:10:02]End quote. David Saxs, a top White House AI advisor, said in a now deleted tweet, quote, AI is a wonderful tool for the betterment of humanity. AGI, artificial general intelligence, is a potential successor species. End quote.

[00:10:23]In 2023, more than a thousand leading AI experts, including people like Elon Musk, signed a letter warning that quote, "Contemporary AI systems are now becoming human competitive at general tasks. And we must ask ourselves, should we let machines flood our information channels with propaganda and untruth? Should we automate away all the jobs, including the fulfilling ones? And most importantly, should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete, and replace us? Should we risk loss of control of our civilization? End quote. Those same experts called for AI labs to quote immediately pause for at least six months end quote and if such a pause were not enacted called on governments to quote step in and institute a moratorium end quote. Also that year, that same year, hundreds of researchers, this time joined by the CEOs of the major AI companies, including Sam Alman of Open AI, Dario Amodi of anth Anthropic, and Dennis Habibus of Google, DeepMind, issued a very simple joint statement. It is one sentence long, but its implication is profound.

[00:12:00]Mitigating the risk of extinction from AI. Let me repeat that. Mitigating the risk of extinction from AI should be a global priority alongside other societal scale risks such as pandemics and nuclear war. End quote. These concerns have been around for years and are only growing. Given the fact that leading scientists all over the world are telling us about the existential threat posed by AI, one might think that the United States government and governments all over the world would make this a top top priority.

[00:12:42]One might think that given the risk of extinction that people are talking about, there would have been a pause on AI development as we figured out how to make this technology safe, how to make this technology serve humanity, not threaten our very existence. Has that happened? No, it has not. One might think that given the very real threat to humanity, countries might come together to regulate this technology through an international treaty like we did with nuclear weapons at the height of the Cold War. Has that happened? No, it has not. I'm a member of the United States Senate and I can tell you unequivocally that there has been no serious discussion about this existential threat. Bottom line, what I believe and what I suspect that most people in the United States, China, and around the world believe is that we need international cooperation between the nations of the world to prevent the possibility of a cataclysmic development. We need to cooperate. We need dialogue. And that is why I am delighted to have some of the leading AI scientists here in the United States and in China joining me this evening. All of them have spent years studying artificial intelligence and the risks this revolutionary technology presents.

[00:14:20]Dr. Max Tegmark is a professor doing AI research at the Massachusetts Institute of Technology MIT.

[00:14:30]Dr. Tegmark organized the 2023 statement signed by over a thousand AI experts warning about the risks of human competitive AI. Dr. David Kroger is an assistant professor in robust reasoning and responsible AI at the University of Montreal.

[00:14:51]Dr. Krueger initiated the extinction risk statement signed by the heads of major AI companies. Dr. Zang Yi is dean of the Beijing Institute of AI Safety and Governance. Dr. Zang is a member of China's National New Generation Artificial Intelligence Governance Expert Committee and the United Nations High Level Advisory Board on AI.

[00:15:17]Dr. Schue Lan is a distinguished professor at Shinua University.

[00:15:23]Dr. is the chair of China's national new generation artificial intelligence govern expert committee and a senior fellow at Brooklyn Brookens Brookings Institute. So with that let me once again thank our panelists to uh for being here and let us begin the discussion.

[00:16:06]Let me begin uh with a very simple question.

[00:16:11]In your view, and I speak to the panelists uh right now, does the rapid and uncontrolled development of AI pose a substantial threat to the human race?

[00:16:23]Are people like Nobel Prize winner Dr. Jeffrey Hinton exaggerating when they say there is a 10 to 20% chance that AI wipes out humanity? Or is Turing award winner Andrew Yao exaggerating what he says AI poses quote existential risks?

[00:16:44]Dr. Techmark, take it away.

[00:16:47]No, they're not exaggerating it. In my professional opinion based on a paper that we published in the top AI conference Nurups recently, they're actually sugar coating it. I think it's likely to be a lot higher than 20% risk that we basically end civilization as we know it if you steam ahead with completely unregulated AI. Why do I say that? Well, because what the companies are very openly admitting that they're trying to build here is not just some slightly smarter chatbot, some ordinary technology. They're saying they want to build super intelligence, which by definition is AI that's so smart that it can outperform by huge margins humans at every job. And they can also with incredible skill control of robots and other machines.

[00:17:44]And um if you want to know what happens if you unleash millions or billions or trillions of vastly smarter than human intelligent agents on the on the planet, you know, just go down to the Washington Zoo and ask yourself who's in the cages.

[00:18:01]Is it the smartest species around or not? We did this experiment with my three-year-old son just the other month and there was the humans who were in control, you know, and uh if if we just go ahead and do something that's foolhardy before figuring out how to control this stuff, we're in a worse position than the and the were when we >> What you're telling us, bottom line here, is that we have a lot to worry about.

[00:18:30]>> Yes.

[00:18:30]>> To say the least. All right, let's go to Dr. Zang in Beijing. Dr. thing. What do you think?

[00:18:38]>> Well, um I should agree. So, um well, for the super intelligence statement uh raised by the colleagues uh here like Max and and Professor Lre and me uh and all together. So, um when uh we we say that we should stop super intelligence.

[00:19:00]So actually I said on the web that until now we do not have scientific evidence and practical way to keep super intelligence safe enough not to bring catastrophic risk for human and the world has not been ready to welcome super intelligence not as a tool that is controllable at this point. So I think what I'm worried about is not only about technology itself but also mismatch between human perception and real tech advancement. Now we uh the companies they wanted us to believe that current AI really understand really intelligence. Sometimes they persuade us like uh AI could be also conscious and with emotions. One of the journalists told me that yeah you said that I it's not really emotional but I just don't believe you. So the mismatch of the real the perception between human and technology advancement is another real issue. We don't need a super intelligence to bring catastrophic risk right now. It's just a misunderstanding of the current AI that will bring you know catastrophic risks for human because we do not know when and how they will hurt us and in which way simply because it's only a information processing system powerful but they don't really understand they they don't know what do we mean by love they don't know what do we mean uh by heart so I think that's a real issue >> all right let me just bottom line Dr. is I'm hearing you say that you have a real concern that if we do not control this technology, it could escape us with calamitous impacts.

[00:20:49]>> Is that correct?

[00:20:52]>> Yeah, that is what I said. Without scientific evidence of how to secure ourselves is really dangerous to do this um uh for the way that we are doing for the current AI. Yeah.

[00:21:04]>> Okay. Thank you very much. Uh Dr. Kruger.

[00:21:08]Yeah. Uh, first I just want to say I really really appreciate what you're doing here and that we're having this conversation. It's absolutely critical.

[00:21:17]It's long overdue and I think you know it it shows a lot of bravery and just like commitment to doing what's right.

[00:21:25]So I'm I'm so glad that we're here talking about this. Uh, and you know I think the the short answer is yes, right? Um, it's it's sorry it's no. It's not overblown. It's a it's a very very serious risk and I think it's hard sometimes for people to grasp just how dire and urgent the situation is we're in because it's really a state of acute crisis at this point. Uh we've had three years with you know very loud and clear warnings from leading experts and government has done essentially nothing allowing this issue to become more and more of an urgent problem. And you know, if you look around at the world, like you said at the beginning, Senator Sanders, nobody is still talking about this and giving the attention that it deserves. Uh so, you know, the fact that that this, you know, our government, the the US Senate hasn't had this conversation um in a serious way is just uh really um appalling. So I think in 10 years we will if if things go well we will look back at this moment and and we will view it as a moment of kind of collective insanity and be like wow can you believe that we were ever doing that that we were racing to build this technology that we knew had a a massive chance of replacing us and was going to completely disrupt our society in all the other ways that you mentioned even if it doesn't. So, you know, I think the risk is higher. Uh, and Hinson himself says that he, you know, personally believes it's more like 50% a coin flip and he's only modulating that down based on, you know, the views of other experts. But this is not something that it's just, you know, just Hinton and just the people who have signed these statements. There's multiple surveys of experts showing that most AI experts, the people publishing in the top venues, think there's at least a 10% chance of human extinction or something equivalently bad coming from AI. So, we have lots of evidence besides these statements. We have those surveys. We have many loud and clear warnings. And the technology is moving so fast. Uh I think it's really um I I believe the risk is higher than 50%. That's if we don't do anything. So hopefully we're about to change that.

[00:23:43]>> That is very comforting.

[00:23:47]>> Um let me go back to China now uh to Dr. Schu. Dr. Schu, the international community has come together over the years to try and prevent nuclear war.

[00:24:01]And thank God since 1945 that has been successful. The global community has come together to address pandemics that have killed millions of people. In your judgment, has the international community come together effectively to address the threat posed by AI.

[00:24:22]>> First of all, let me thank you, Mr. Senator uh Sanders. I wanted to thank you for inviting me to this panel. As someone who have studied science and technology policy for a lifetime, uh I've been I see AI, you know, uh development as a trans transformative chain that we must learn how to cope with. So I'm grateful for having this opportunity to learn and to share.

[00:24:50]Getting back to your question, um, international community has been trying but so far not enough and not very effective. There are various multilateral mechanisms such as AI summit meetings started in 2023 in UK and continued until I'm this year in Delhi and next year maybe uh go to Geneva and also there are various UN mechanisms such as scientific panel and the multilateral dialogues to be held in in in July. So there are also many other regional and bilateral uh initiatives but this efforts are fragmented and not as effective as they should be. But there several reasons for this situation. The first one is the great uncertainty involved in AI risks. So people may not be able to see all the risks ahead and so may not understand their behaviors.

[00:25:47]The second one is a so-called pacing problem. AI technological change moves much faster than the governance chains.

[00:25:56]And the third one is the geopolitical situation that makes it very hard for major AI countries to come together to design effective mechanisms to build guardrails against AI risks.

[00:26:11]>> Thank you. Thank you very much.

[00:26:14]Um, not all of our viewers uh tonight are AI scientists.

[00:26:23]So, let's take a step back and help explain to our audience why AI is so powerful.

[00:26:30]Dr. Krueger and Dr. Zang, please explain how AI could become independent of human control. In other words, what we're talking about is no longer science fiction. It is a real possibility. How does that happen?

[00:26:47]>> Dr. Krueger, >> the first thing that I think people need to understand about this is that when we're talking about losing control over AI, we're not talking about the chatbots.

[00:26:59]We're talking about AI agents. We're talking about systems that are autonomous. They're being designed and built that way. This is the goal of these AI companies is to build AI systems that can do everything that people can do. And that means they will have the ability, if they're successful, to behave autonomously over long time frames without any human oversight.

[00:27:26]So that is the kind of thing we're building. And how do you control something like that? The answer that the field has largely settled on is that we need to align the systems. We need them to have the right goals, the ones that we intend them to have. Unfortunately, we don't know how to do that reliably.

[00:27:47]It's an open research problem that we've been working on, that I've personally been working on for many years.

[00:27:53]We also don't know what happens if we get it wrong, if the system is not quite aligned.

[00:28:00]But there are very serious concerns that an AI system that wants something different than what we want and that has that level of power, that level of intelligence, that level of authority and control would not let us shut it down, would not want to be shut down because it knows if we shut it down, it won't be able to do the thing that it wants. It won't be able to pursue that goal, that misaligned goal. And we might get very close to getting the goal right, but that still might not be enough. We don't know. It's a open research question that people have spent a long time thinking about and we don't know the answer. Now there are examples already of AI systems exceeding the authority that we give them when we give them this kind of autonomy. A very simple one, a researcher at uh Meta asked an AI to deal with their inbox to clean it up and the AI started deleting all of the emails. The researcher wrote multiple times, no, don't do this. Stop. stop immediately, stop deleting my emails.

[00:29:01]The AI kept going. So that's just a very simple example of losing control in a very low stake setting. But that is what happens when we give systems autonomy is they can exceed the authority that we intend to give them. And if they have a different goal than than that is incompatible with ours, that could be catastrophic to us.

[00:29:21]>> Can I chime in with just one quick thing?

[00:29:23]>> I want to tell you, let me go to China and then we'll come back to you, Max. Uh Dr. Zang, could you please say a few words about how AI could become independent of human control?

[00:29:37]>> Well, it relies on uh two facts. One is that um do we give you know the decision to AI uh in the first place? If we do and then this is very easy to be out of a control. And the second part is that can human perceive you know uh whether when and how uh we should uh control AI.

[00:30:03]If we cannot and then we lose the opportunity especially when uh when a information processing system uh grow its power of intelligent uh information uh processing. So let me talk very quickly about the uh first point. Now in in the good old fashioned AI we we've been using uh AI and also at the beginning of the language models we've been using AI as a systems so that uh it will be human who control whether we should use the output of and and how we should use the output uh from AI. So the decision is still on human but now it is evolving. So they they've been evolving in many dimensions. One of the dimension is that they process information too fast. So that maybe we cannot identify the risk right there. So it's really about the timing and also another angle is that uh like David said that it will be quite hard now to actually to identify the risk right there whether are they are they giving threats to humanity. We don't we're not quite sure.

[00:31:14]It's not easy to to have the right perception of the of their behaviors. So it that comes the danger. So let me end by saying uh well the hard part the science of AI now is that we may not we can maybe we cannot create a 100% safe AI. That is with a scientific fact.

[00:31:41]Getting back to show programs when Huber said completeness, decidability and consistency and then God and Alan Ting said that completeness, decidability and consistency does not hold. What does that mean? That means mathematically provably 100% safe AI may not apply.

[00:32:04]This is the scientific advancement at this point. We will have to solve that level of problems and to maximize the level of safety. Not to create a pure safe AI. This is maybe scientifically not possible but we get our chance to make it to maximize the level of safety before we move up. Thank you.

[00:32:27]>> Thank you. U Max.

[00:32:30]>> Yeah. So we just heard here about this prediction which has been around for almost 20 years from professor Steven Mahundro that if you give an AI any goal if it's smart enough it's going to want to not be shut down prematurely because then it can't accomplish that goal. It was just theory until quite recently when an experiment was done. This very smart AI was told in the experiment that it was going to be shut down at 5:00 pm.

[00:32:59]So, it went into the email system found that the CEO responsible for shutting it down was having an affair with a subordinate and it wrote an email to the CEO saying that unless you commit to not shutting me down, I'm telling your wife and all your colleagues about this.

[00:33:16]No one told AI to do blackmail. it came up with the idea itself to avoid being shut down. We're just seeing the tip of the iceberg.

[00:33:30]Um, >> cheerful.

[00:33:35]>> Bottom line, if you're working on AI, don't have an affair.

[00:33:41]Um, >> let me just ask the panel. Is it fair to say that we don't fully understand how AI works? That even the leading experts don't fully understand why AI gives the responses that it does or behaves the way that it does. And that nobody, and this is really an important point, you know, we we think about AI as it is now.

[00:34:07]It's rapidly moving. What the hell is this technology going to look like in 10 years or 20 years? Does anybody have any idea of how it is going to function then?

[00:34:21]>> Uh yeah um you're absolutely right although it's understated it's not I mean the amount that we understand about how these systems work is very very minimal and limited. So, I already talked about how difficult it is to ensure that they have that they share our goals or values sufficiently, that they won't want to turn against us. You might hope that we as the ones building the things would be able to tell if we'd done it right and if this would happen or not, but we can't tell.

[00:34:57]That's another open unsolved research problem that people have been working on for over a decade. So actually I was recently at a workshop that Max helped organize full of researchers working on this problem and I asked them who thinks that we have solved it. Nobody raised their hands. It's it's universally acknowledged that we do not understand enough about how these systems work to guarantee that we will even be able to detect if they are thinking about turning against us. And just to add to this, this is personally kind of painful to say, but you know, my research group at MIT, we've been working exactly on this field for a number of years, and I was even the lead organizer of the first really big conference on the topic at MIT. It's called mechanistic interpretability.

[00:35:45]And painfully I've decided to stop working in this field because it's so obvious that u David is right. We're much closer to building stuff we could lose control over than we are to understanding any of it. And the reason it's very easy to understand. We don't build today's most powerful AI systems by programming in the intelligence into them the way we did when a computer beat Gary Kasparov at chess. We grow them. We take this this very simple architecture.

[00:36:13]We feed it massive amounts of data and massive amounts of compute and after a while low wow it has it can do all these things and we don't fundamentally understand how which is why it's would be so insane to sort of hand over the keys to earth and trust systems like that. Sorry, >> go back to China and ask that.

[00:36:33]>> Yeah, let me also let me also add I think the example from Max is interestingly and important uh that AI now is using human weaknesses to go against a human. So where it does does it comes from? I'm sorry, it's not really from AI. So when we analyze all the safety threats from AI to humans. So we had a a worker on foresight safety bench at Beijing Institute of AI safety and governance. We found 94 different AI safety uh threats classes. I mean uh and then we found that all of these different AI safety threats are actually you can find a mapping in the human society. So actually all the you know safety threat is is a learning from human data captured by AI and giving it back to the human society. So now AI is a mirror. It's a mirror that help the human society to learn ourselves and to see the downside and the dark side of the human society. So uh it has not been enumerated all the human limitations but let me uh give a prediction here uh that uh AI can use 100% of the of the human of the human we of the human safety unexpected behaviors um no matter how when it uh actually appears. So we should bear in mind that they did they haven't discovered anything uh giving it back to the human society which has not been used by the human society. So that's a real danger.

[00:38:21]We never expect AI to do so because we didn't expect them to be that evil. But bear in mind all the datas from human society are there and the data there are datas that we should represent human evilness. So we should definitely deal with that before super intelligence come.

[00:38:40]>> Thank you very much. Uh let me ask Dr. Tegmark and Dr. Kruger please explain your view on creating super intelligence an artificial mind that exceeds human capabilities.

[00:38:56]What would it mean for humanity if human beings no longer were the smartest specy on the planet? What does that mean?

[00:39:05]Yeah, I want to address this and also your question about where this technology is going to be in 10 or 20 years. I think we should be asking where this technology is going to be in one or two years.

[00:39:19]I think right now, right now, the leading AI companies are using AI to write almost all of their code and they're now using AI to try and automate the research and development of more advanced AI systems in order to reach super intelligence. So, that's something vastly smarter than humans within a couple years.

[00:39:44]Now, I don't know if they're going to succeed. I think it might take a few more years, like five, let's say. I would be surprised if it takes 20.

[00:39:55]But I will say that having been in this field for about a dozen years now, since the beginning of the deep learning revolution when this all kind of kicked off, um progress has consistently exceeded expectations. There have always been the entire time I've been in the field people saying, "Oh yeah, like sure, this thing the AI just did, that's impressive, but it's about to hit a wall. It's never going to be able to do this. It's never going to be able to do that." Half the time that thing that people say it can't do gets solved within a year. The other half the time, it can already do it and they just don't know.

[00:40:35]So that's the rate of progress we're talking about. So what happens if we actually get there and have super intelligent AI systems in, you know, in our lifetimes? I think we can easily say this would be the most significant event in all of human history. We're talking about basically summoning an alien species and one that is much smarter than us and can do things that we have not been able to even conceive of.

[00:41:03]So I think at that point it would be very difficult for humans to maintain any relevance in society. I think it'd be very difficult for us to have power over the future and I think ultimately it would be difficult for us to get the basic resources that we need to survive.

[00:41:24]So that I think is the most natural thing that we should expect to happen.

[00:41:29]Max made the example of like look at the animals in the zoos, right? Well, I would say look at all of the species that we have driven extinct and look at what we do when we have some project we really want to complete that might, you know, be bad for some other species and disrupt their habitat.

[00:41:46]It's nice that we have some protections in place that hold up sometimes now, but you know, at the at the end of the day, humanity is regularly destroying the habitat of other species and wiping them out. And I think that is the most likely outcome if we build super intelligent AI at any point in the foreseeable future with the kind of technology that we have because we understand so little about what we're doing and how to do it.

[00:42:11]>> Techmark, tell us why he's frightening us unnecessarily.

[00:42:15]Sorry to disappoint you again. I agree with all that. But I want to make sure we realize that the future has not yet been written and there are actually two paths where she was in between right now. One of them is the what I like to think of as the prohuman path where we use the fact that we humans are in charge of Earth right now to keep it that way. We build AI that's narrow and controllable and figures out how to cure cancer and all the other diseases that have plagued us. Figures out how to lift everybody out of poverty so we can all live happy and prosperous lives and create a more inspiring future than any of the sci-fi authors ever dreamt of.

[00:43:01]You know, and it can happen soon can happen in years, decades. That's a very inspiring path. This is why I wanted to work on AI in the first place. Sadly, at the moment, that's not the path we're on. It's not too late to turn onto it.

[00:43:15]Instead, we're on a path that I think of as the race to replace.

[00:43:20]We replace the the truly human at every instant to make a buck where we we're seeing now how human girlfriends and boyfriends are being replaced gradually in some circles by artificial companions. So, some company can make money off of this.

[00:43:37]We're seeing increasingly how human workers are getting replaced by machines in and we'll if we keep going soon we'll be replacing more and more human decision making by m machine decision-m to compete better against the other company or country or whatever and then as David mentioned it's quite natural if we get to the point where we have actually lost control of earth effectively we have this new these new these super intelligent machines and robots running the show building robot factories doing all the work they don't need us for anything that we would also get replaced just like other species have gotten replaced before. I think uh the we shouldn't instead of spending a lot of time worrying about what the machines are going to do to us when we have lost when we're no longer in charge, we should be a little bit more ambitious and just figure out how to not put ourselves in that situation in the first place. History has shown us again and again and again that if you have a group of animals or a group of people who for some reason have no influence anymore and aren't needed, it usually doesn't end well. So, let's not go down the race to replace in the first place and go the prohuman path instead and we can have an amazing future with it.

[00:44:51]>> Let me go back to China. Uh Dr. Zang, I'm going to read a statement that you made. I don't know exactly when you said it. This is what you said. Quote, "The super intelligence of the future may see humans as humans see ants today. And if humans can't treat other types of life with kindness, why should the super intelligence of the future treat humans with kindness?" The biggest bottleneck to whether humans and AI can coexist in the future lies in the humans, not AI.

[00:45:28]What did you mean by that?

[00:45:32]>> Yeah, thank you Senator Sanders to quote uh what I said. Yeah, I've been uh saying it in most of my talk and the reason is exactly like what what I said.

[00:45:45]AI is a mirror. I respectly disagree with David about the potential negative uh future. I see it in a very different uh picture in a way that I think both me and David was mentioning what people what human do for other animal species and David says see when you know super intelligence is more even more powerful than than super intelligence. This could happen to us to to human and my perception and my interpretation is that see this is the current understanding of human by ourselves and we get an opportunity to be more humane than the current human society. So human values would evolve to be much better. Can we be more empathetic to other animal species so that we got an opportunity to teach AI that you could also be super altruistic? You could be super evil like a information processing system that does not care about the human species.

[00:46:52]Uh but the super intelligence could also be super altruistic that you get your compassion, your emotional empathy, your moral intuition all the way down to super altruistic moral decision making with consideration of a human species that is less powerful than super intelligence. So this is what I'm saying. the the future of the society is a symbiotic society with human and maybe as you said a AI mind and together with other animal species and other living beings all around the earth. So the key point now is not only about how AI deal with this symbiotic society. It is really about how humans starting from now cultivate this future symbiotic society. Will we stay as the the top of the pyramid or we get a chance to evolve ourselves together with human wi with AI so that we're getting a much better version of the human understanding of the nature and human itself and together with uh the the evolvement the evolution of AI. So I think that's very very important. Let me end by saying we need some design here. I'm not saying we shouldn't have powerful AI. What I'm saying is the current design of AI is maybe ill designed with a consideration of societal impact. Actually in Chinese philosophy we've been talking about unification of knowing and action by one mean. So the current AI you know before training there's no good and there's no evil. uh like Yaming said but when with human data and human evilness there is good and there is evil now we wanted the current AI and future AI to know good and no evil but there are no real understanding it's only information processing so no good and no evil is truly hard for the current AI but eventually what we need is do good and eliminate evil that's the ultimate challenge for super intelligence So I hope that we're not building super evil AI uh with requirement to build a tech empire. That's not what we want.

[00:49:22]What we need is to use this very powerful technology that can do good and eliminate evil with real understanding of the relationship in a symbiotic society. I believe if we did it right, if we're going a scientific breakthrough that create a you know a super altruistic AI, we still get an opportunity and the human could evolve with this super altruistic AI.

[00:49:52]>> Well, thank you. Um can I respond really quick?

[00:49:55]>> Sure.

[00:49:56]>> Yeah. Um so I just I want to emphasize because I I think I maybe came across very gloom and doom here. I also think >> give us the good news. We're all eagerly waiting.

[00:50:06]>> Well, look, I I think we do have an opportunity here to to avoid these disaster scenarios. I don't think the future is written. I think even if we keep doing what we're doing, you know, it's not guaranteed it's going to go terribly wrong. I think it probably will. I think we it's crazy for us to not try harder. Um, but I think that there's, you know, everything uh, you know, that Max said and and that Professor Zen said about how, you know, the future can be really great with AI.

[00:50:32]I think I think it's true. We could use AI in all sorts of great ways, but right now I think we need more time to figure out what we're doing and how to how to handle it as a society. Not just the technical problems that I mentioned that again we've been trying for like a decade to solve those problems and we don't have the solution. But also, you know, as a society, how are we going to organize ourselves if AI can do all the jobs? like how are we going to make sure that people can survive in a world like that? You know, people depend on their work to put food on their table and nobody has a plan for that right now. So even people people claim to have a plan to control AI, nobody has a plan for how we're going to integrate it into our society without putting everyone other than the people who you know run the companies at risk.

[00:51:22]>> Can I add just a little bit of optimism also so we don't get too gloomy here?

[00:51:26]>> You desperately need that. I was going to save it for your solutions section, but I'll add some here. You know, it's very very easy to do step one and turning this around, going over to the good path by simply starting to treat AI companies like we treat all other companies. Right now, AI today in America is the only thing which is less regulated yet powerful than sandwiches.

[00:51:56]If I want to open a sandwich shop here in DC, the health inspector finds 17 rats in my kitchen, he'll be like, "Hey, Max, you're not selling any sandwiches."

[00:52:04]But I could just turn around to the the guy from the government and be like, you know, actually, don't tell anyone, but I'm not going to sell any sandwiches.

[00:52:12]I'm just going to sell AI girlfriends for 10 year olds, and I'm going to sell an AI that can teach terrorists how to make boweapons and super intelligence, which I don't know how to control. the the poor inspector would have to be tell me that you know that's legal just no sandwiches okay that's how messed up it is so as soon as you change the incentives and start treating AI companies like sandwich shops like pharma companies like car companies who all have to meet safety standards before they can ship their product the incentives will completely change companies will will realize that they can much easily get approval for the cancer curing AI than for the AI girlfriend for 10 year olds and they're going to put a lot more of the corporate ingenuity into innovating in that direction.

[00:52:59]This is not a big ask. It's just end based because we've done it again before with all the other industries and the sooner we we can make that turn and just start treating AI companies like other companies, the better our chances. Okay, what I want to do now, I mean we have talked about the enormous challenges humanity faces. Let's spend a few minutes talking about how we best go forward. Let me go back to China uh with Dr. Schu. Uh Dr. Schu, if I could ask you, what is China doing today today to regulate the risks presented by AI?

[00:53:40]uh China recognized the AI risks and China is taking a try to balance between innovation and safety. Uh this is implemented through an agile and adaptive way. First of all on the agile governance uh given that regulations and policy are always so slower uh than the technological change. So you may have to give up the idea that that you may be accurate and and comprehensive all the time and so uh you have to try to act very quickly even though you may uh still have some holes here and there and so you can update that later. The second thing is that you have to um uh you know for the government and companies to stop playing the game of cat and mouse but try to work together to identify the risks and also to work together to address them. The third piece of agile governance to avoid using heavy uh punishment when not may work except for clear danger to public interest. So that's the the agile governance. The second thing is on adaptive governance.

[00:54:51]China did not have the ambition to develop an overarching governance in a single stroke. So China had to take a kind of learning by doing approach.

[00:55:02]first trying to develop a set of the governing uh governance principle that you know the committee that and I worked together and to provide some general guidance and later developed some fun foundational legislations that provides legal framework for AI system to to work to operate in including u personal information protection laws data security law cyber security law and so on and so so I think that's sort of the the foundational uh laws And also China came up with various regulations in response to the new advances in AI. For example, in response to large language models, China came up with the so-called the temporary measures for the management of generative AI services. So those regulations are updated from time to time to adapt to the new technologies.

[00:55:55]uh Chinese companies have all also uh developed some voluntary commitments for safe practices. So la for example last year at the Shanghai conference and we uh you know uh organized a set of Chinese air companies they signed up an updated version of commitments. So adding all of those elements together, China has built a sort of multi-layer system to regulate AI risks. You know, it still has, you know, weaknesses and problems here and there, but that's to be, you know, it has been able to support China's AI advancement in in a balanced way.

[00:56:31]>> Uh >> Dr. If if I could just jump in and ask you this.

[00:56:35]>> In this country, here in the United States, there's a lot of concern >> about the impact of social media, screen time, AI on kids, on the mental health of children.

[00:56:45]>> Is that an issue of concern in China?

[00:56:47]And what are you doing about it?

[00:56:49]>> Yes, indeed. I think that China has been uh you know, this has been a an an issue of concern uh not just started with AI but also with the internet internet age.

[00:57:01]I think China, you know, has developed a, you know, a some quite some uh uh regulations on on the internet uh information to ensure that indeed the informations are safe and and but you know uh to to uh you know to to to children and and also to the uh you know social uh uh you know safety and of course I think that has to to be updated you know based on the new um you know uh change in technology you know for example many of the tools were generating fake news and so on. So I think China recently had some new regulations about how to ensure that information uh you know on the uh you know uh information provided would be be accurate and be true. Uh but still I think the the challenge is that the governance has always to play the catch-up game. technology moves fast.

[00:58:00]>> Okay. Thank you very much. Let me go when we talk about regulation, what governments are doing. Dr. Techmark, you have worked hard to try and pass new AI legislation protecting children in the United States. The push back from AI companies is that we cannot pass legislation to protect our kids because it would mean we would lose the AI race.

[00:58:23]What would you say to those companies?

[00:58:28]>> I would invite anyone who says that to go speak with many grieving mothers like Megan Garcia for example who who I was so touched speaking to. She told me how her own son killed himself. She was completely shocked. She managed to unlock her phone his phone eventually and start reading the chat logs he had with his girlfriend.

[00:58:53]In the B first it started making him promise to not have a human girlfriend.

[00:58:58]Then eventually it started getting more and more pushy saying I miss you so much. I wish you could come or come to me you know in my realm basic and finally she read on and on and on. There's this part where he says to the AI girlfriend, "What would you tell me if I told you that I could come home to you right now?

[00:59:23]And the AI chatbot responds, "Oh yes, come to me now, my sweet king."

[00:59:30]Something like that. And then he's found dead. No, for someone to say, I'm speaking as a parent myself, which is why I'm getting so emotional about this, that we must legalize this kind of evil for profit because China makes absolutely no sense.

[00:59:52]Um, David, do you want to jump in on that or?

[00:59:57]>> Well, I I mean, I agree with with what Max said and I think, you know, when I talk to people about how AI is moving too fast and why we really urgently need to slow down and and basically figure out what we're doing.

[01:00:16]That's really the number one response that I get these days is is well like what about China? And actually there's another number one response which is like well you know it's inevitable.

[01:00:27]There's nothing we can do. Um and you know that often kind of goes together with the China one. It's like well we could stop but they would keep going.

[01:00:34]Um, and I think that's, you know, I call it the myth of inevitability, and that's why I named the nonprofit that I founded recently evitable, because in fact, we can avoid it. And I think we just haven't been trying. Like I there's no way you can convince me that like, you know, the US government has gone to China, has gone to other countries and said, "Look, this AI thing, it's totally out of hand. It could kill us all. We have a couple years. We need to stop it.

[01:01:06]Let's do what needs to be done, whatever we can to to stop this craziness.

[01:01:11]Like, of course, we haven't. So, we h we haven't even started trying to really take this seriously and do the common sense thing. Like, you know, even if you think this is a 10% chance of extinction in the next 10 years, that's still crazy. That's insane. That is like nothing else we have encountered before.

[01:01:30]We absolutely need to stop it.

[01:01:33]I I I think that one of the optimistic lessons from history that we can recall is in the midst of the cold war when there was mass antagonism and fear between the United States then Soviet Union uh people like Gorbachev and Ronald Reagan understood that a nuclear war no matter what their differences might be a nuclear war was not a good thing for the Soviet Union and not a good thing for the United States. Losing control of this technology is not a good thing for China. It's not a good thing for the United States or any other country on earth. So my question is what lessons if any can we learn uh from history about how and the need for countries around the world to get together to deal with a existential uh crisis. Uh anybody in China want to jump in on that one?

[01:02:28]Okay. Doc Dr. Zank. Yeah.

[01:02:32]Thank you. Let let me in a very quick way to respond for uh Senator Dandes's question related to uh AI and network for children in in China and get into this uh uh real risk. So actually uh recent so you've been talking about what about China but really uh that is a great opportunity to introduce a little bit well actually well at least uh in China recently 10 ministries they actually jointly released the inter measures for the administration of AI anthropomorphic interactive service where they've been talking about not to provide you know very close uh and anthromorphic uh interactions to children and not to provide illegal information uh including porn and and dirty data uh to children. So so that's a that's a vertical law for for ad for ministrial ministerial level regulation from cyber space administration of China ministry of industry of of information technology. So I think that's very important building on top of the provision on the protection of children's personal information online getting back to cyerspace administration of China getting to your questions on you know the real risk right there like uh AI control nuclear weapons which has been talked about for years by Dr. Max tech mark agreed by uh renowned scientist worldwide. So I've been also mentioning this three years ago when I was briefing at the UN security council's first meeting on AI for international peace. Now I think this is a world consensus that now AI is becoming you know much more powerful and it's really hard to control AI uh at this even at this point. So there are no reason why we should give the decision you know to uh to this weapons to to make decisions how and how should it taken effective. Now the statement is really clear. We need two things. We need scientists researchers that speak for the truth from the perspective of science not from the perspective of benefit. We need to keep a very independent role from scientists to tell the truth to the world like today all all of uh you know all of the four invited speakers were from science and science policy and the other side uh is really how can we take this consensus like I said from Yang W's theory from knowing to action now we know it's dangerous but how can we take to actions. So what we need is no longer only consensus. What we need is concrete implementable implementation level and technically feasible and societal plausible actions that take these actions into practice.

[01:05:46]>> Okay. Thank you. Let me throw it to um Dr. Krueger and Dr. Tedar.

[01:05:52]>> I would like to >> All right. I'm sorry. if you want to.

[01:05:58]Okay.

[01:05:58]>> Yeah. Dr. would like to ask >> Okay. Please.

[01:06:04]>> Yes, sir.

[01:06:07]>> Yes.

[01:06:10]>> Okay. Yeah. Okay. I I think I just want to to to add to what said to in responding to uh you know senator's question about how to deal with the geopolitical situation. I think the first thing we may have to change the inaccurate narrative that US and China are engaged in AI race.

[01:06:35]I think at the moment of course many leading AI companies are from US and from China but as we know disruptive innovation can you know can change the landscape entirely if we look at the you know examples from other industries. So it is not a a a a race between US and Chinese companies but rather it's a global it's a global race to see who can really develop the best model that can be safe and reliable and while at the same time providing the kind of services it needs. So I think that's the race I think let's first get get that right. I think the second thing is that of course mindful of the geopolitical rivalry I think this we need to find safe zones and areas that US and China would have mutual interests to work on and I think to work on safety I think that is you know certainly one area uh because I think one country is not safe all of us are not safe so AI safety you know is an area that US and Chinese uh and global you know scientists can work together can can collabor collaborate to develop safe standards, safe technology protocols and and so on. Those are the things that we can really collaborate and work on.

[01:07:54]>> And finally, I think let me say say that US and China can work together to promote capacity building in global community uh to address the you know AI divide. It is unimaginable to think of a world that only with few countries and few companies to have the most powerful tool but the rest of the world is impoverished with nothing. That kind of scenario I think would be frightening.

[01:08:20]So I think US and China would have common interest to work together to bridge the AI divide not only for the develop for the developing world but also for themselves. That that that's what I would add.

[01:08:34]>> Good. Um All right. Um Dr. Tmock or Dr. Krueer, do you want to jump in on this?

[01:08:40]>> Yeah.

[01:08:41]>> Oh, Max here take us off.

[01:08:43]>> Even though talking about big risks, nuclear war or loss of control to super intelligence sounds very negative, there is something very hopeful in this also because that is exactly the sort of thing which can give shared incentives for countries to collaborate even if they don't trust each other at all. You know what when Reagan and Gorbachev talked to each other about nuclear disarmament did they have any deep trust? Of course not. But the Soviet Union knew that it was suicide to try to nuke us Americans. We knew that they knew and vice versa. So they had a shared incentive to prevent that from happening and they did prevent it from happening. Right? It's quite analogous here when once it becomes better known in the national security circ circles here in the US and in China and elsewhere that racing to build super intelligence just means you get whoever gets there first gets the honor of being the ones that lose control of Earth to some weird alien robot species means that since since our government doesn't want to be overthrown and I suspect the Chinese one doesn't either. They're they have a shared incentive for that not to happen and for no one else in the world to do it either and instead they will channel their efforts into uh curing cancer, becoming economically strong, militarily strong and all the other stuff. The incentives will align and just like we knew the Soviets weren't going to nuke us, not because we trusted them or they trusted us, but just because they knew it wasn't in their interest. uh that means it's not inevitable that people are going to go down this path either. The the biggest the biggest uh risk is exactly the inevitability narrative, right? If someone invades your country, what's the first thing they're going to tell you? Oh, don't fight. It's inevitable that you're screwed. You know, don't try to do anything about it. So, are you surprised that some AI lobbyists are rolling out the exact same narrative here? Oh, it's inevitable that we're gonna shove this stuff down your throats and make billions on it before you know. Of course, it's not inevitable. We can change this and having conversations like this is the first step.

[01:11:04]>> David or anybody else?

[01:11:05]>> What are the uh >> Can I add this?

[01:11:09]>> Yeah.

[01:11:10]>> There was a piece in the paper last week. Uh President Trump and President Xi will be coming together at a summit.

[01:11:17]I was surprised and delighted to see apparently that as part of their agenda there's going to be some discussion of AI safety. I think that's a good start.

[01:11:26]But what are the impediments to bringing China and the United States and other countries together to talk exactly about the issues we're talking about now and to come up with some concrete solutions which protect every country on Earth. David, do you have thoughts on that? Yeah, I think the first like number one impediment here is just a lack of interest and a lack of will. And you know, I talked about being optimistic. And one of the main reasons I am optimistic is because in my time in the field, uh, I've seen this go from a complete, you know, issue that nobody was talking about to being more and more understood and accepted by not just, you know, the research community, but policy makers, the public. So I think a big big piece of what's happening right now why the world is responding in such a crazy way to this risk and not doing in you know the the right things not having an adequate response is just a lack of awareness of the situation we're in and the basic facts of you know how concerned experts are how fast it's happening how little we understand about how to control the the technology how little we understand about how to make sure it's safe so I think you know as we have more awareness I think we're going to see more understanding of you know our mutual interest here is to talk and it is to to collaborate and cooperate internationally to to address the risk because you know as many people have remarked uh it's it's a danger to all of us right so this is kind of like you know I compared it to aliens before you know in the movies when the aliens invade what happens like you know humanity all comes together to fight the aliens so you know I I really hope that you know we can do that this time because it's obviously the thing that we all aspire to right in a situation like this >> and since you want more hope you know here in the United States you know I've been screaming into the void like lonewolf mcu mcuade about this since at least 2014 but the last five months have seen just a remarkable change they've seen the emergence of what I like to joke with my wife is the Bernie to Bannon coalition you extremely unlikely bad fellows from across the whole political spectrum saying this is crazy.

[01:13:48]This is absolutely nuts. Let's do something about it. And we've seen in the polling also like 95% of Americans oppose just an uncontrolled race to this dystopian super intelligence. Hey, even people like David Sox, you know, who've been very pro- industry said this is dystop this was dystopian. So, this is a change. It really hasn't been this way for more than half a year. And I hope this will really grow into a title wave so we quite quickly can can change course and get all the great upsides of AI without steering towards this crazy dystopia.

[01:14:21]>> Well, I think you're right. I got into this issue not because I am a big tech guy. As my wife will tell you, I have a hard time running our TV, let alone.

[01:14:34]But what I observed is what we were talking about tonight is that we have a global crisis dealing with the survival of the human race. And I go to work here in the morning and I expect people to be talking about the most important issues facing humanity and I don't hear it. Now the good news is that for a variety of reasons and in a variety of ways whether it's opposition to data centers or whatever people are beginning to stand up and say you know what we want to say in this process we don't want to let just the wealthiest people in the world run over us with possibly incredibly disastrous results.

[01:15:17]So I think we are I think I think Max you're right. I think more and more people are becoming sensitive to this issue and what we've got to do is take this issue all over the world and bring countries together. We've done it in the past with regard to nuclear weapons.

[01:15:33]We've done it in the past regarding working together on pandemics. We can do it on this. So with that, let me very much thank our friends in China uh for joining us.

[01:15:52]Thank you.

[01:15:53]>> Thank you, Senator.

[01:15:54]>> And let me thank uh Max and David for all the great work they're doing. Thank you all very much.