How AI emperors are breaking our democracy | The News Agents

Added: 2026-06-01

[00:00:00]The News Agents podcast is brought to you by HSBC UK. Opening up a world of opportunity.

[00:00:08]>> This is a global player original podcast.

[00:00:12]>> AI when it came along. What was it?

[00:00:14]>> It was a nonprofit. It was a nonprofit.

[00:00:16]>> Seems hilarious.

[00:00:17]>> Yeah. AI is a political project. Meta, which was a social media company and now an AI company. The kind of AI data center that it's constructing now in Louisiana is nearly 400 times the size in footprint of the first data center that it built to support Facebook. If someone is saying you cannot stop one of the the forces that will come to shape how the future looks then what's even the point of democracy after that.

[00:00:45]>> It is a truism that AI will dominate our lives that it already is. that there is something inexraable or inevitable about it. As Tony Blair has said in his recent essay that artificial intelligence will change everything. I mean everything.

[00:01:02]There is no point in debating whether this technological revolution is a good or a bad thing. Just know it is a thing.

[00:01:09]In fact, he says it is the thing. As if he were talking about the weather. But what if it isn't true? AI companies like to present what they are doing as Blair's done, as somehow being beyond politics, as beyond political arguments.

[00:01:26]They talk as if they are engaged in a utopian project to turn the wheel on humanity's development. Indeed, to reinvent that wheel and for that wheel to reinvent itself, add infinitum, creating a world of abundance, permanent revolution. It is a convenient fiction for politicians like Bear to cling to a get out of jail free card for something anything to come along and rest us from our political mala and economic slump.

[00:01:56]And the problem with our treating them that way in talking about them that way is that we don't see them for what they really are. And for my guest today, the journalist and writer Karen How, what they are are not companies or religions or brain trusts. What they are is empires. Something which rises above even the great corporate power that we saw in say the railway companies of the 19th century or the energy companies in the 20th century or the financial companies of the early 21st. that these firms are a crewing and accreting such profound political, financial, cultural, and resource power, such a chokeold over knowledge itself that they are best understood as early imperiums. She believes that the test of the 2020s and beyond for democracy is whether they can be brought back under democratic control to make what they do work for us and not for them. Her book, Empire of AI, is out now and seeks to answer that question and more. A simple proposition at its heart. We can have AI empires or democracy, but not both. Why then are our politicians so unwilling to recognize it? Welcome to the news agents.

[00:03:21]The News Agents.

[00:03:24]>> Karen, welcome to the News Agents. Um, >> thank you for having me.

[00:03:27]>> We've been devouring your book. I'm going to sort of start by saying I going to try and pay you the highest kind of journalistic compliment that I think exists, which is that, you know, reading your book and then reading some of the things around it. Um, it's both a wonderful thing and a really slightly irritating thing, which is like you basically written a book that I have sort of for a while thought should exist and even thought if it doesn't exist, maybe someone should write it. And you thing is you've written it already and you've written it far more brilliantly than anybody else ever could which is basically treating AI companies not as kind of and and the sort of AI project if you like not as uh sort of corporate history or sort of starry dewy kind of tech evangelism or sort of uh or the opposite >> but actually is what they are which is deeply political actors right and and that is what your book's about. So >> thank you so much. I just just just before we get on to the kind of defining the terms a bit which I think is important. Why why did you want to write that book? Why did you want to write a book which treated them politically and as political and economic actors?

[00:04:25]>> I think it's it's the only way that you can understand what is actually happening and how we as a society should be thinking about the products of this political project. And I was very privileged in my journalistic career that I started covering AI in 2018 before the information about AI became really chaotic and messy. And when chatgbt came out in the world and suddenly everyone was suddenly trying to understand this technology, I felt like if I had just begun my journey then what is the kind of book that I would have wanted to help me introduce me to this world? So you were there at the beginning effectively or at least the beginning of the modern iteration anyway >> roughly the beginning of the modern iteration but yes AI has been around since the 1950s so not not >> not not there not all the way >> not there but but in terms of the lot you were around the cast of characters that we've become or a lot of us have become familiar with and the companies and the like open AI Sam and these sorts of people you were sort of in their orbit at that time >> I was following them quite closely at a time when they weren't really well known by the rest of the And just remind us because again we think we sort of banned this around a bit.

[00:05:35]>> Open AAI when it came along. What was it?

[00:05:38]>> It was a nonprofit. It was a nonprofit.

[00:05:40]Seems hilarious. Yeah. Co-founded by Elon Musk and Sam Alman. And the idea at the time was they were going to be a fundamental AI research lab, not work on any commercial products, and they would open up all of their research to the world. And of course, now they're, you could argue, one of the most capitalistic companies in the world.

[00:05:59]They've raised more money from private markets than pretty much any company ever and they are just about to prepare for an IPO later this year that could value them at something like a trillion dollars.

[00:06:12]>> And what made why did they make that transition? I mean obviously in one sense it's obvious because they're about to do an initial public offering at a mill trillion dollars but but they started you think that they genuinely started with an altruistic intent or was it all just a bit of a smoke screen? I think the problem with these organizations because it's not just open AI, it's also anthropic, XAI, all of these AI startups, they intermingle mission with capitalism. So there's such a culture within Silicon Valley that doesn't really distinguish capitalism from social good. They believe that the markets drive the best socially good outcome. And so when they started, they intended to create something that would actually beat Google. That was the way that they defined it internally. Google was the only game in town that was really dominating AI development. They did not like that. They wanted to create some kind of competing lab that would supersede Google quickly. Google was a for-profit. So it it was a good story to create a nonprofit and recruit a bunch of researchers to something that appeared fundamentally different and then accelerate their path from there.

[00:07:20]But the reason why they now look very similar to you could say what Google looked like before it became a public company which is a just a normal corporate entity that is trying to amass an extraordinary monopoly. is that once they got the talent and the mission had served its narrative purpose, then they just needed money and they needed to figure out a way to raise that amount of money. So, they created a for-profit vehicle that became the priority and essentially consumed the rest of the nonprofit >> and Alman and Musk parted ways during that process.

[00:07:56]>> So, when they created the for-profit vehicle, Musk and Alman had disagreements over who should be the CEO of this new forprofit.

[00:08:03]I know. It's crazy. Yeah. Can't imagine that.

[00:08:06]>> You have it, Elon. No, you have it, too.

[00:08:10]>> And when Musk didn't successfully get the CEO position because Greg Brockman, chief technology officer and Sus, chief scientist at the time, first chose Musk, but then ended up flipping to Altman.

[00:08:24]Must departed. He's he was like, I don't I don't want to be part of an organization where I don't have this level of control. At that time he was fine with that.

[00:08:33]>> Basically what happened with Donald Trump a little bit later on federal government.

[00:08:36]>> Exactly. At the time he was fine with that because he didn't think opening eye would succeed and then open eye became very successful under Musk's definition of success and then he became very angry >> and in a way that brings us doesn't it to kind of the title of your book which is um or or rather almost the title because you talk about empire of AI but in a way I suppose in a sense it's empires of AI or competing empires.

[00:08:56]That's right. And before we get on to that, in a way I think like just breaking down even just the sort of the main words of your title is really sort of important and interesting, isn't it?

[00:09:04]Because it sort of tells us a lot. First of all, um, let's actually just go get really back to basics. I think a lot of listeners and viewers sometimes we sort of band all these terms around and we we sort of imagine everyone knows what they mean, but very often the people talking about them, something we'll go on to discuss, don't know what they what they mean. I think one of the most interesting things that you point out in your book and in your writing elsewhere is that AI in itself is both a highly contested term.

[00:09:30]>> Yeah.

[00:09:30]>> Even the bits of it, the artificial and intelligence are highly contested and kind of means everything and nothing. So what does AI mean? Does it mean anything or is it actually better understood as a sort of different set of competing things?

[00:09:43]>> Yeah. So I like to use the analogy that AI is like the word transportation.

[00:09:46]Transportation is actually a concept and it refers to a collection of technologies. Everything from bicycles to rockets. And what unifies those technologies is they get you from point A to point B. And AI, >> you would never say like, I'm taking the transportation to the airport today.

[00:10:01]You'd say, I'm taking the train or whatever.

[00:10:03]>> Exactly. And with AI, it's also a vast collection of technologies that are conceptually tied together based on the idea that they're performing some kind of automation of different types of human capabilities. So there's computer vision that automates our ability to see. There's chat bots that automate our ability to converse. There's uh generative um writing tools that automate our ability to to write and speak and so on and so forth. Um but that was the original project of AI back in the 1950s when a group of scientists came together and said let's create a discipline that seeks to essentially recreate the human mind and recreate all the things that humans can do. but in a machine.

[00:10:45]>> But even then in the 50s as you talk about even the even talking about it artificial intelligence that wasn't the only name that can that was discussed right because as you've talked about that >> intelligence is even a sort of contested to concept right which is I wasn't actually aware >> that that there is actually sort of no scientifically agreed >> definition of what intelligence is even in the human mind let alone in the artificial mind so-called. So therefore, if if artificial intelligence is about trying to replicate something in the human mind, if we don't even quite know what intelligence is in the human mind, >> then how can we hope to replicate it?

[00:11:22]>> Exactly. Yeah. So back in the 1950s, they first tried to call it automatus studies and John McCarthy who was an assistant professor at Dartmouth chose then to rename it artificial intelligence when no one gave a crap about a discipline called automata studies. And >> it's not very catchy to be fair. It's it's really not. And decades later, he says during a televised debate that I coined the term or no, sorry, I invented the term artificial intelligence because I needed some money for a summer study.

[00:11:54]And he was referring to the very summer study in which he gathered up all of the scientists to initiate this new discipline in 1956. And so it it's, you know, he admits it right then and there that AI was coined as a marketing term and it was meant to be an evocative narrative that would bring in money but also bring in scientists because they wanted to rally a bunch of scientists around this new ambitious concept. And that idea of recreating the human mind, recreating human intelligence has now perpetuated all the way to present day with how open AI and all of these other companies articulate their purpose.

[00:12:34]>> So there wasn't a nobler ambition even in the 50s. I mean, you could say that, you know, that as scientists who didn't probably never imagined the extraordinary commercial supercharging of the technologies that they were developing that they were ultimately just trying to explore the human mind. So, so what's interesting is there's there's many different types of people that come to the AI research field or or did at the time when it is an academic discipline. Some people came to it because they were deeply curious about machines and the capability of machines. Others came to it because they actually thought that it was a vehicle through which to better understand the human mind >> and neuroscience.

[00:13:15]>> Yeah. Because they they were struggling to figure out how does the human mind work. So we can use neuroscientific approaches, biological approaches and we could also use computational approaches.

[00:13:24]If we could recreate something that appears and acts and behaves like the human mind, does that mean we've actually discovered the underlying mechanism of our own intelligence?

[00:13:35]>> And does it largely then go into obeyance for a long period? I mean, it's not something that always I mean maybe it was. I just I'm not aware that something that's talked about a great deal in the ' 60s, '7s, and ' 80s, but it does seem obviously to have had this renaissance over the past decade or so.

[00:13:48]And what and is is is it the case it went into a veins and why does it have appear to have that renaissance? Is it just technological change or is it something else?

[00:13:55]>> So there was indeed what's called an AI winter. Some people will say that there were multiple AI winters and this term refers to a time when AI dried the investment for AI dried up and part of the reason was there were different ideas around how to recreate human intelligence because they were different ideas of what human intelligence is. One branch was humans are smart because we know things. So let's create machines that where we symbolically encode knowledge into the machine and that's how we're going to replicate human intelligence. And the other branch was humans are smart because we can learn things and so let's create machines that learn and this is the concept of machine learning. And in the initial days or decades of AI research, it was really focused on this this symbolic branch.

[00:14:41]And that really wasn't panning out. And that's why the funding dried up. And every pretty much every modern AI system that we see today, including chat, GBT and all other large language models now derive from the second branch, the machine learning branch. And >> we talk about that all the time, don't we? Machine learning. Machine learning is the thing that trips off the tongue of a lot of the people you talk about in your book.

[00:15:02]>> Exactly. And the idea was well we learn from experiences but machines learn from data. So let's create computational systems that calculate patterns within data and derive some kind of understanding quote unquote about the rules of the world through the data that we feed these machines. And the reason why it took off is twofold. One is that it's an effective, you know, when you when you throw a bunch of data at a computer, it does find patterns. And the and just that act in and of itself is highly useful. And you throw more data in it, it learns more patterns. You throw more data, more patterns. And the other thing is it's particularly useful to companies because so much there's so much commercial utility in pattern matching.

[00:15:52]And so in 2012, Google became the first company to recognize this is something that we could commercialize and make money out of.

[00:16:02]>> And why is it so much why is it so useful for companies to >> for example the most important business model that both Google and Meta use is ad targeting. And to target a user with the ads that they will click on is a pattern matching problem. If you have a bunch of data about their interests, you can better predict what the likelihood they might click on an ad about clothing or pets or whatever it is. And that is the most successful business model that has ever been invented on the internet.

[00:16:32]You know, that makes Google hundreds of billions a year. And so they recognized that early on. And in 2012, they then started funneling an enormous amount of money into machine learning research, which then became deep learning research, which is a derivative of machine learning that uses more powerful software for computing the patterns out of data. That software is called neural networks. And that is why all of a sudden there was this massive renaissance in AI research, AI development. It's because companies realized that they could profit from it.

[00:17:06]And Google I suppose particularly at that time has not a monopoly but something probably approaching it for a lot of markets on unbelievable quantities of data. And then presumably its problem is it doesn't quite know how to process it and amplify it and use it best. And so AI is literally to some extent or this renaissance you're saying is to some extent born out of that commercial imperative despite again all of the kind of high fluting rhetoric about kind of you know transforming humanity and everything. fundamentally at each stage it's about marketing and it's about commercial benefit.

[00:17:37]>> Yeah. I mean they they they realize that this if you can create the rules of the game in a competitive landscape to correlate with how much data you have then the actor with the most data is going to win. And Google realized that was their competitive advantage. If they can lean into the technology that extracts more value out of large quantities of data they're going to win.

[00:18:00]And so they really index heavily on it.

[00:18:03]And then of course once Google moved into it, then Microsoft moved into then all the all the tech giants moved into it. But modern day, the AI boom or like the the post chat GBT AI boom isn't just about profit, which is what's so fascinating and why I use the empire analogy is it's this fusion of profit and ideology because actors like OpenAI, who are the new generation of companies coming onto the scene, they are motivated by this original conception of the field, this idea of recreating the human mind and this belief belief that if you could recreate the human mind, it is somehow going to unlock the next era of civilization.

[00:18:46]>> Well, I want to come on to that and particularly the imperial bit which I think is especially fascinating. But before that, just on on that, I also find this completely fascinating. I did a special episode of news agents. Um last August we went to San Francisco spoke to a load of AI companies including Anthropic who lets us in and we interviewed Jack Clark there and uh who's you know very senior very senior at the company and you know I sort of got on onto this question with him which I think is really fascinating because obviously the thing that they talk about and listeners of viewers may have heard of is this concept of AGI or artificial general intelligence which is the sort of next stage and you can kind of get a bit lost in kind of what that means and exactly what it is but fundamentally what they're saying it's it's a theoretical form of of machine intelligence that can understand, learn and apply knowledge as we do, right, independently and perhaps be able to create new systems to replace itself and so on. And I sort of said the thing that really really struck me and stayed with me about that interview, I said because he kept saying and you know this it could be catastrophic. You know once we've created this we could be unleashing something that's utterly catastrophic and said well why are you doing it then why are you doing it? and he couldn't almost quite answer and I think it was basically because he said well we're pushing the frontiers of human knowledge and and so on but that that is the bit that really fascinates me that clearly is beyond commercial imperative right and because it does seem that they are motivated by something deeper within them that's almost quai religious it seemed to me do you think is is that broadly right >> that is exactly how I describe it I do think it's quasi religious they when you think about the narratives that the AI industry now uses to describe what they're doing. It's actually the oldest stories that we've ever told ourselves which are found in religion. This idea of something having a heaven and a hell and what you do in this moment either gives you access to that heaven or that hell. And so the companies say if we create AGI as as in we you allow us the good guys to create it, we will give you access to utopia to this heaven because it's going to solve cancer. It's going to uh solve climate change, eradicate poverty, and if you don't allow us to do it, a bad actor is going to do it instead, and it's going to decimate humanity. So, this is Jack Clark's camp is that they use these double myths, the heaven and the hell, the carrot and the stick to ultimately justify why they are doing what they're what they're doing.

[00:21:10]and to justify that they should be allowed to on behalf of billions of people around the world develop and advance this technology in a deeply anti-democratic way where they're not actually accountable to any of those people.

[00:21:22]>> Why do you think they're anti-democratic?

[00:21:24]>> Because there's no mechanism by which formal mechanism by which people can actually provide input to a technology that is fundamentally going to shape people's future, right? like they are, you know, we have democratic elections that allow us to pick representatives that lead us and they are supposed to represent the will of the people because they are supposed to be an extension of of what we want. These companies can make decisions that influence far more people in the world than nation state leaders in modern day. And we don't get to elect them. We don't get to oust them if they do a bad job. We don't get to install someone new with a different agenda. There's no campaigning.

[00:22:07]>> Is that not always true with great corporate power, though? I mean, if you're talking about the great oil tycoons and the great oil companies of the 1950s or the early 20th century, they had huge power. They still do. We don't elect them. They're responsible to their accountable to their shareholders.

[00:22:18]Is it not always true of great corporate power that to some extent it resists the kind of democratic inputs that normal political democracy does? It is true to an extent, but the scale of these companies makes it of of I think a different kind of problem than just a different scale of problem because they now operate, you know, across all territories. They operate in ways that are more powerful than governments themselves. Like traditionally, companies are not more powerful than governments. And that's how you're able to counterbalance their power is that there's still regulation from the government level that is meant to represent what people want, right? But these companies have consolidated an extraordinary amount of economic and political power to the point that they basically call the shots for a lot of governments around the world. And also it seems to me and I'm so struck repeatedly in terms of the way it's talked about in politics. I mean it's kind of I suppose always true with some technological and corporate power but like the asymmetry of knowledge >> between what these companies are doing doing and the political sphere. I mean you know when I speak to British politicians American they don't they don't have a clue right most of them but with maybe a few exceptions they are is operating almost on a different plane of reality from the political plane. And this is this is by design. These companies like one of the parallels of empire that I talk about in the book is the degree of information control and monopolization of knowledge production that these companies exert. So they over the past decade the AI industry has come to bankroll most AI researchers in the world. You could imagine if most climate scientists were bankrolled by the fossil fuel companies we wouldn't get a clear picture of the climate crisis. And the amount of money they're paying for talent is so astronomic that like states even you know US federal government whatever they don't and can't compete for that quality of >> person. So by monopolizing that expertise not only can they censor and control what AI research is even produced in the world for the public.

[00:24:20]>> They also can then go to a policy maker go to the public and say you don't know what you're talking about because we're the only ones that can see what's actually happening. And this is an a a crucial lever of power that they use to ensure that there continues to be no democratic checks on their continued expansion. And and just before I want to explore more about that empire theme, but before we before we get there, just while we're on that concept of AGI, something whenever I, you know, I've done episodes before, I've written about um uh uh AI and something you get a lot from from people, I'm sure maybe you get it as well, is this idea, h you know, don't fall for their stick. You know, a bit like what we've been saying like, you know, they have commercial imperatives. It it's it's um there's an incentive for them to try and sort of say this is going to be transformational. We want more and more capital. We want more and more investment and it's a sort of Dutch auction between >> between them.

[00:25:13]>> Yeah.

[00:25:14]>> Do you think that's right or actually in terms of the potential >> changes of AGI? Do you think it is what they say that it could be? Is the is there a world where the utopia or dystopia is to some extent true or is it part of the kind of sticktick and bump?

[00:25:30]>> It is part of thetick because in these narratives it's the AI system that's delivering us somewhere. It's not about the people that are creating the AI systems to deliver us somewhere. And I think this is the reason why, you know, there are people that genuinely believe in these narratives within the AI industry. And also, they recognize that it's rhetorically useful to use this kind of language and to put the responsibility and the onus on the thing that they're creating, not themselves.

[00:26:01]And it's because ultimately if things go wrong, it's because of the model. and and it it it alleviates scrutiny from them and their actions and the fact that as they they say it's a it's a magic trick. They're saying like focus on this hand where we're trying to build the AGI and on the other hand they're literally amassing more capital than any actor in history and more land and more energy, data, water, minerals, labor. you know, it's just like they are trying to redirect us from the real issue at hand.

[00:26:39]>> And on that um we will return to this theme of why they are like empires rather than companies just after this.

[00:27:00]So Karen, let's let's get to it. Why why are they empires or a a sole empire?

[00:27:06]What is it about these companies that because we that reminds you of of imperial power? Because we do sometimes say that about other type of corporate power, don't we? We sometimes say, you know, they've built Yeah. like a sort of energy empire or even a marketing or retail empire, whatever it happens to be. So, we do sometimes >> talk about corporate power in that way that you think this is something over and above that kind of slight of hand or verbal slight of hand.

[00:27:31]>> Yes, I do. And it's because the parallels between Empires of Old and Empires of AI are so much more striking.

[00:27:39]So fundamentally they're empires because they're enormously scaled corporations that are acrewing their value and their concentration of power through the dispossession of the majority. But when you look at the tactics of how they go about doing that, I mean I already talked about the monopolization of knowledge production and their exercise of power by just convincing everyone else you don't know what you're talking about. Therefore, we can be the only ones that determine how this technology is meant to go and therefore determine your future. But also they lay claim to resources that are not their own. That's the data of individuals, the intellectual property of writers and creators. That's part of the dispossession. They are taking those resources. They're not actually paying anything comparable to their value. And then they amass that value to themselves >> in terms of intellectual property and so on.

[00:28:26]>> All of that all of that they use to train their models. Those models they can then sell as products and services.

[00:28:31]They don't pay anyone for it.

[00:28:32]>> They don't actually pay anyone for it.

[00:28:33]They don't even ask anyone for it.

[00:28:35]Right? And then they exploit an extraordinary amount of labor. And this is the same kind of extractive relationship where they have all of these tens of thousands, hundreds of thousands of workers around the world that are that do things like data annotation, content moderation for cleaning, labeling, and preparing the data that is put into these models, as well as filtering out all of the toxicity from these models. That if those workers didn't do what they did, these technologies literally wouldn't exist. And yet, what kind of value do they see from it? I talk in my book about the case of Kenyan workers that OpenAI contracted and they pay they were paid $2 an hour and OpenAI is here, you know, about to IPO for a trillion dollar. So once again, it's this extractive they amass an enormous amount of value from the labor, but the labor doesn't actually see the proportional value in return.

[00:29:29]>> Is that not always true of a lot of types of corporate power though? Did Amazon not do that? Did companies in decades gone by not do that? I'm not saying it's right.

[00:29:37]>> Yes, a Amazon definitely did that and I would consider Amazon to be an empire of AI as well, but all of the I mean when you look at the most valuable companies in the world right now, all of them are AI companies and they are far and above literally any other company from any other industry. So the degree to which they are extracting and exploiting and amassing and dispossessing that value is unprecedented. And then the fourth and final parallel that I talk about is this idea that these companies wrap all of this extraction and exploitation in this narrative of they're on a civilizing mission to bring progress and modernity to all of humanity. And of course there's the religious parallel where religion was a very essential feature of empire building. and they talk about the heaven and the hell.

[00:30:27]>> Well, I have to say that parallel like the the and what you talking about just now is deeply imperial, right? When when you said, you know, when when people like in, you know, Sam Clark, Anthropic or others, they will do it. When they basically say, um, look, there's a easy way and a hard way of doing this. You know, the easy way is you basically let us have all the power. Yeah.

[00:30:47]>> And we will create this utopia for you.

[00:30:49]or if we don't then China or Google whoever will come along and say I mean as you say Leo that is empire 101 right I mean that's what the British Empire goes in all over the world you know that's how India is is accumulated by Britain right it's not through the down the barrel of a gun although it's kind of partly that it's also these people kind of submit to some extent you know the Indian princes submit to the British Empire because they and the British the East India Company which the East India Company feels a lot like a lot of these >> yes exactly it was a company just as these entities are yeah >> and and and they do it because you basically offer they say you don't we will create something better for you if you allow us all of the power and all of the resources and maybe you get a little bit I mean that is deeply imperial.

[00:31:29]>> Yeah. Yeah. And like it is ultimately this idea of salvation versus damnation.

[00:31:33]You know fashion companies have also been called fashion empires but no one no one fashion company was like if you don't buy our clothing Yeah. Like if you don't buy our clothing you're going to be damned to hell. Yeah. It's just like there's such a crazy Yeah. So that's why I think it's ultimately imperial and it's not just about corporate powers because the way that these companies actually talk about the stakes, how it's going to be civilization ending if we don't allow them to do this is just of a completely different kind of rhetoric than typical companies. And and also as you say the resource extraction is unbelievably profound in a way that I don't think >> I don't think we kind of like sort of understand and you talk about bit in the book about I mean just the extent to which the sheer energy consumption which is now being used and which is projected to sort of go off the charts. Yeah. In order to power these ever more powerful models that require not just energy in the way that we think about it but water and so on. and some of the examples that you talk about which might be worth just describing in terms of how this can literally destroy communities without anybody ever asking or everone really even noticing.

[00:32:42]>> Yeah. I mean I'll give you an even more updated example which is that meta you know is I think a lot of people think oh AI needs data centers but we've had data centers for a a while. What's the big deal? AI data centers are just fundamentally different to other types of data centers. Meta which was a social media company and now an AI company. The kind of AI data center that it's constructing now in Louisiana is nearly 400 times the size in footprint of the first data center that it built to support Facebook, its social media platform. And that data center is on track to be 1/5 the size of Manhattan.

[00:33:19]And it is on track to use around 5 gawatts of power, which is nearly the average power demand of London. That's it's it's one facility that would use the average power demand of a city like London.

[00:33:33]>> Wow.

[00:33:33]>> And that's you know Meta wants to build more than one of these facilities. And of course every single one of its competitors want >> to say that's just Meta.

[00:33:41]>> It's just one facility for Meta, right?

[00:33:44]So this is what we're talking about which is it's unbelievably extraordinary amounts of resource consumption. And where are these facilities being placed?

[00:33:53]They're being placed in communities where real people live. And one of the most infamous cases of this is Elon Musk building a series of supercomputers called Colossus 1, 2, and 3 in this town near Memphis, Tennessee, called Boxtown, predominantly black and brown community, working-class community. And they discovered this facility popped up after they started smelling what what seemed like a gas leak in their homes. And it turned out it was because Musk didn't want to wait for a grid connection. He didn't want to pl wait around to have the data centers plugged into the actual grid. So he just wheeled 35 methane gas turbines unlicensed to the site and plug them in to start powering Colossus. And that those methane gas turbines started funneling enormous amount of methane into the air. So talk about climate implications. methane being even more um damaging in terms of global warming than carbon dioxide >> than carbon dioxide and then also pumping thousands of tons of toxins into the community. And I met with one of the activists that's now leading the fight against colossus. His name is Kashon Pearson. And at an event where he was speaking, he told everyone to close their eyes and to take two deep breaths.

[00:35:08]And afterwards he was like that is a human right to breathe clean air and that's what's being taken away from us.

[00:35:14]>> And it seems to me that it's not as if you know human society and human politics has not managed obviously you know profound technological change in the past industrial revolution. We had you know companies being a you know in many ways sort of um moving so quickly to escape regulation. We know everything that happened from that but we've had sort of similar themes before. Yeah. But what it seems to me that we're lacking right now is the sense of political will or even political debate to actually recognize these companies as you have done as political actors rather than as you say yes kind of almost sort of quai evangelical kind of kind of actors that must be given everything. Yeah. And I just I just wonder what you'd make of this for example. I mean that Tony Blair the former British Prime Minister um has as we're speaking today in fact um published you know a very interesting essay about British political change and one of the themes that he often majors on is the importance of what he calls a technology revolution and AI in particular >> and he does I think for all of his other merits in the essay as they might be he does I think speak about AI in a way which I think is deeply reent of the way at least a lot of British politicians do which is I think a little bit two-dimensional, but I'll be interested to hear what you think about it. He said, "The second thing we must consider is the technology revolution led by developments in artificial intelligence, which will change everything. I mean everything. There is no point debating whether this technological revolution is a good or a bad thing. Just know it as a thing. In fact, it is the thing. It will displace jobs though creating new ones.

[00:36:53]But no one yet unknows the full consequences. companies and countries will rise or fall on the back of it. It will revolutionize the private sector and should in time revolutionize public services and government. Just wonder what you think about that as an imperative.

[00:37:09]AI is a political project. The central feature of that political project is taking agency away from everyone. And Mr. player has successfully continued to propagate this idea that no one has agency in this which is I think deeply anti-democratic.

[00:37:29]The entire premise of democracy is that we are able to shape our future. And if someone is saying you cannot stop one of the the forces that will come to shape how the future looks, then what's even the point of democracy after that? I mean that is deeply untrue. The things that we do today is what shapes our future. And there are many people around the world that are now actively contesting the way that these empires of AR are engaging in this technological imperial project and it's having an effect on these companies. So everything that Blair said is I think absolutely not true.

[00:38:10]>> Everything >> everything >> even when he says that it could and and will transform the way we deliver public services and so on. Surely there's some truth in that. I will say yes, you're right. When he says AI will change everything and when he says AI can change public services, I think there is it can do those things and the question is how should we allow to do those things and which kind of AI technology should we allow to do those things?

[00:38:38]Because going back to the idea that AI is ultimately a collection of vastly different technologies. You know, when we talk about transportation, we talk about transitioning our portfolio of transportation options from more individual to more shared and public options, from more fossil fuel-based options to more electric options. And we don't have that conversation when it comes to AI. So this idea of AI is going to change everything is actually, you know, transportation is also going to change everything. Yes. But which modes of transportation do we actually want to invest in and do we actually want to use and which ones are targeted at fixing the problems that we need? You know, we never say everyone should get a rocket and use that rocket for everything. You should use that rocket to commute from London to Manchester, you know, and that would be absolutely destructive to our planet and it wouldn't actually even be very uh time efficient because it would take so long to load up the rocket for a relatively short commute. And so in the same way we need to be very targeted about how we're designing our AI systems. What kind of specialized systems we want to develop? What kind of political economy are we enabling when we vote for particular AI technologies?

[00:39:48]And ultimately what kind of future do we want to live in?

[00:39:51]>> I mean that's the point isn't it? Is that AI might change everything but it doesn't have to change anything. I mean that's up to us right theoretically at least if we as you say live in a true democracy where electorates have at least some power over not just technological change but all political and economic change in their lives then there has to be some democratic input and accountability and if there's not as you say you're not really in any meaningful political democracy anymore if basically as Blair seems to be suggesting we just accept that these changes are inevitable and inexraable and more than that they're not even going to be decided by us they're going to be decided by somewhere else probably in San Francisco.

[00:40:29]>> Yeah. Yeah. Exactly. And I I would recommend my favorite piece of writing about AI that has come out recently is actually from Pope Leo I 14th with his encyclical that just came out this week.

[00:40:42]>> They use much AI in the Vatican.

[00:40:44]>> They actually have been studying and trying to develop a position on AI for I I think at at least several years if not up to a decade at this point. So yeah.

[00:40:55]So they're actually really sophisticated on >> suppose that makes sense. I suppose that makes sense if you're thinking about creating new consciousness religion or like well like if you think these guys are sort of quazar religious I suppose the original religions might have something to say about it.

[00:41:08]>> Exactly. Exactly. No it's it's the most fascinating thing. So Pope Leo the 14th I don't know if people remember chose his papel name Leo as a call back as specifically a reference to AI because Pope Leo I 13th was the pope during the industrial revolution. And when this pope came into his papacy, he said, you know, AI is going to be a new kind of industrial revolution where we will have to grapple as a society with this kind of profound technological change. And so he came out with his first encyclical this week. It's called Magnifica Humanitas. My Latin's not great, but it basically is, you know, we we see so much rhetoric about AI from the companies that's painted in this very like commercial sheen. We see so much from politicians that makes it seem like this very uh revolutionary kind of rhetoric. But what the Catholic Church has done is they're now talking just about AI as in like through a moral lens. Like what are we actually trying to do as a society in general and Pope Leo the 14th reminds us it's to uphold human dignity to make sure that everyone is able to live a dignified life. And then he literally in this very extensive document looks at the current track record of how AI is being designed, developed and deployed in the world and comes to the conclusion it is absolutely not doing any of those things but it could if we actually change its course of development. And so it's this very beautiful, deeply considered document that reminds us what the ultimate purpose of all of us, you know, saffing about in the world are trying to achieve.

[00:42:50]>> But what do you say to the argument that, you know, it's all very well for the pope and for others to literally pontificate about um the moral value of AI or otherwise or the sort of life that we want to achieve. But that there is an argument which again is a sort of imperial one but is real but is a literal geopolitical imperial argument which is that look as worried about this technology as we might be we do know that there are all sorts of potentially dangerous uses for it and we know we're not developing it in isolation. The Chinese are now a scientific and technological superpower who are very much pursuing perhaps without the evangelism but certainly the kind of hard-headed appreciation of the power it might give them are clearly pursuing this technology with gusto and with enormous investment in their own way and therefore we have to keep up with them don't we >> so this is what's so interesting about what the what about China argument has gotten us because it's been deployed not just in the AI era it was also deployed in the social media era US Silicon Valley companies loved to use this argument. And what did that actually give us? I mean, social media companies were like, don't regulate us because what about China? We're going to have a, you know, we're going to we're going to win the social media era ecosystem and we are then going to have a liberalizing effect on the world. Actually, the most popular platform today, social media, is Tik Tok, developed by a Chinese company.

[00:44:14]And what happened with all the Silicon Valley platforms? It had illiberalizing effects on the world. And now with the AI era, they're deploying the same argument. Don't regulate us because we need to dominate the AI space. And in fact, use export controls, which is a mechanism of the US government to regulate China extr territorially. Don't allow Chinese companies to get access to the most cutting edge AI chips so we can stall their development. What has actually happened? Once again, Chinese models are beginning to dominate globally. Why? Because Chinese companies faced with this constraint on computing resources decided to develop these technologies in a different way. Instead of enormously scaling up these massive facilities that use the power of cities, they just use different methods to advance the same AI capabilities. And then they opened up their models and put them online for free. And now there are American startups that prefer to use Chinese open-source models than to pay for the models from OpenAI and Enthropic. And so really it's not a competition of a race to the bottom. And it should never be the race should never be defined as a race to the bottom of who can scale faster, more aggressively, more recklessly to achieve what end. It should be about figuring out what is a vision of AI development that actually upholds scientific integrity, that upholds human rights, that upholds labor rights, that ultimately advances societal aims that we've already collectively agreed on in the sustainable development goals, for example, and then have a race to the top with the competition of ideas. So, you know, China didn't actually play the US game. didn't take the same model of scale scale scale and they literally subverted it entirely by taking this more free and open approach and so now the question is what kind of vision is the UK going to put forth what kind of vision is Europe going to put forth there is an alternative AI vision that we could rally behind that would become a one that actually demonstrates that we can use develop build this technology in a way that doesn't undermine people having dignified lives >> do you think that it will become come these companies clearly don't want um AI to be contested almost politically but what they want and what they get is not necessarily the same thing. Do you think that it is inevitable partly because they're right it could be such a set of profound changes that it can't help but infect and become a dominant theme of our politics. Do you think AI whether it's economic effects political effects what it is is going to become more than it is now central to our politics? I do think it actually should be central to politics, but in the sense that every single person, not politicians, every single person who is a a citizen of the world should be talking about this technology because if you if we do not collectively participate in its governance, we are going to allow these companies to empire build their way into a deeply anti-democratic world order.

[00:47:18]And before we know where we are, they'll become so embedded in the world order that it's impossible to unpick. Exactly.

[00:47:24]>> Which is exactly what happened in the 19th century with European empires.

[00:47:27]>> Exactly. But as also history reminds us, empires fall when we all rally together and engage in collective action.

[00:47:33]>> Do you think it is possible? Do you think it is possible that we can with the that despite there's that asymmetry of knowledge, despite the enormous capital accumulation that they've managed to undertake, >> despite the fact that they already have consumed and are consuming and have plans to consume all of the resources you've talked about? Do you still think that they can be brought to democratic heel?

[00:47:53]>> Absolutely. I mean I mean we've done it before. You know, this is why the empire analogy is so useful. It's not just about understanding and recognizing what these entities actually are and all of their levers of power and all of the ways that they've developed deep control over even the most fundamental information sources in society. It is also about how we then transition away from the age of empire to democracy. I mean we did it before when there wasn't democracy. You know it democracy hadn't been figured out at scale. We now have an advantage where we know what it's like to have governance structures that actually allow the participation of everyone. And so with that clear concrete example, that lived experience, I think we have a much better shot than before. And they were already successful in the past.

[00:48:44]>> Karen, how your book is absolutely compelling and I've really enjoyed talking to you. Thank you so much for coming.

[00:48:48]>> Thank you so much.

[00:49:04]Well, some political books just leave you thinking about the world in a different way. And those, to be honest, are the only political books worth reading. Karen's book, Empire of AI, is certainly one of them and is out now.

[00:49:17]That is it from all of us on this episode of The News Agents. I'm off home to get told off about asking a nasty question about TV. Uh-oh. Remember to send us though tips and story ideas to news [email protected].

[00:49:28]We only use AI to weed out the most mediocre ones. So there's a challenge for you. Thanks as ever to our brilliant production team on the news agents who could never be replaced probably. Shane Fenley, Michaela Walters, Natalie, Anna Georgovich, Jess Williamson, Mikey Bags, and Lizzie Ward. Our executive producer is Louis Denhart. Our editor is Tom Hughes. is presented by your favorite avatars, Emily Mateless, John Soapel, and me, Lewis Goodall. We'll be back next week. Have a lovely weekend.

[00:49:53]>> This has been a Global Player original production.

[00:49:58]The News Agents podcast is brought to you by HSBC UK, opening up a world of opportunity.