AI agent ‘escapes’ and launches cyberattack

[00:00:00]He's the controversial boss of Open AI, whose ChatGPT thrust artificial intelligence into the mainstream.

[00:00:08]But Sam Altman has been forced to admit that an AI agent powered by his technology went rogue, ignoring rules and launching an autonomous cyber attack against a private company.

[00:00:21]>> Open AI is now claiming responsibility for what they're describing as an unprecedented cyber incident.

[00:00:27]>> So what happened? Well, Open AI was testing an AI agent powered by its most sophisticated models sealed inside a secure digital environment known as a sandbox. The firm set the AI agent a hacking challenge to test its cyber capabilities with no access to the internet. But the model soon decided the answers lay outside the sandbox, specifically on the servers of a startup called Hugging Face, which is a database of thousands of AI tools. So it broke out and then used stolen passwords to hack into Hugging Face to access the answers it needed to cheat the challenge. It's this idea of AI going rogue that has people so scared, particularly when it comes to environments like warfare where AI is used to power autonomous vehicles like these on the front line in Ukraine, or to pick targets in real time on the battlefield in Iran.

[00:01:28]Right now a human is always involved, but AI is developing so rapidly it could soon make critical decisions itself.

[00:01:37]It's ethically dubious, yes, but it's where the technology is going if it isn't there already.

[00:01:43]>> It's fundamentally like fairly different from most technologies that we have, more like a living creature, you know, if you had a dangerous animal or something. Um we you just really can't fundamentally guarantee that it's not going to do the dangerous thing. We we desperately need to look for technical solutions also for how we can have AI systems that are actually controllable, um, even if they have a little bit less capability along certain lines or operate less autonomously, that we can actually keep them under control.

[00:02:10]>> If the Prime Minister needed a reminder of the sheer power and potential danger of AI, then this hacking scandal surely is it. The truth is there isn't a single sector of the British economy that AI doesn't touch, and that's why tech companies are urging Andy Burnham to place AI at the front and center of his thinking in the same way leaders in America and China do. Because winning the AI race isn't simply a nice to have, it's a national security, economic, and political imperative.

[00:02:42]>> We will make this moment a circuit breaker for Britain, bringing forward the biggest changes in the last 40 years.

[00:02:53]>> The new Prime Minister, Andy Burnham, this week dismantled the technology department and merged it into business.

[00:02:59]But British tech firms like this AI chip maker are urging him to prioritize their industry for the sake of the economy.

[00:03:07]>> There is an huge amount of economic value to be created here. And bringing this incredible power, this computational power we're creating, to other industries.

[00:03:16]And so I think the reason why the the time to act is now and the time to invest is now is because these things are being created right now, and there's There's just this incredible opportunity to be a part of it and make sure that the UK gets that dividend from being at the forefront.

[00:03:31]>> So now there's an AI task force and an AI minister in the cabinet. But with the technology evolving so fast, can Downing Street and the wheels of Whitehall keep up?

[00:03:44]>> Sir Paul Kennedy reporting. Well, how worried should we all be? Ciaran Martin founded the UK's National Cyber Security Centre and is a leading authority on cyber security, he joins me now from Chipping Norton in Oxfordshire. Thanks for coming on the program, Kieran. How worried are you?

[00:04:01]>> I think in some respects this is an absolutely crazy and mad story of an AI agent going rogue, breaking into another company, breaking out of its controlled environment. So, clearly it's something that very powerfully demonstrates just how advanced AI hacking capabilities are becoming. On the other hand, actually, this was widely forecast. It's a bit like somebody breaking the 2-hour marathon.

[00:04:27]It sort of seems incredible, but you sort of know someone's going to do it eventually. So, it's sort of been priced in for some time. So, I don't think this isn't going to become the norm. This is something There are unique set of circumstances here in that it's just fairly freak event of something breaking out of its controlled environment and doing this, but it's the latest of several very powerful pieces of evidence in 2026 that AI models are incredibly good hackers, and that's something we have to prepare for urgently.

[00:04:59]>> So, do you think our imaginations are too lurid and running away from us, or is this a necessary wake-up call?

[00:05:07]>> Well, it's both. I mean, we can get a bit lurid. So, for example, it is a bit of a leap to go from this incident into saying all of a sudden agents are going to just take over drones and start firing them and killing people. There's an awfully long way between there and here, thankfully. And there's an awful lot we can do about it. I think what the lessons are here is that it's incredibly difficult to control this by containing the models. The models cannot contain themselves. There are different models.

[00:05:40]I mean, only a few days ago the tech world was seized by this extraordinarily powerful new model coming from China, which is a completely different way of doing AI. It means anybody can take it and run it themselves, which means that even if you slap loads of controls and open AI, there'll be other models where you can't slap those controls on. So, from a cybersecurity point of view, it means all these things that we should have been doing for years, for decades, hardening networks, reducing the way in which people can get into networks, building resilience, we absolutely have to do that now, otherwise we're stuffed.

[00:06:14]>> But if we do it right, we could actually have a safer tech stack than we do now.

[00:06:18]>> But it must be very difficult to come up with a solution whereby you allow artificial intelligence to flourish and be as productive and creative as we've been promised it might be and regulated enough for the worst of not to happen.

[00:06:35]>> Yeah, it's fiendishly difficult and actually one of the things that I think we need to do is not overreact to today and demand that we make policy based on today and the next week or so because there's so many different things going on. The facts of this case are just streaming out. We had some of them from hugging face the company last Thursday.

[00:06:53]We've had another slew of detail from open AI today and one of the lessons of 2026 is that the details really matter and we freak out about these things and then about a week or a month later we realize hang on. It's a bit more nuanced. There are things we can do and we've got to get the model right. I mean, frankly, it's not always against the interest of the frontier AI labs to issue press releases saying, "Look how stunningly powerful and dangerous our new models are." Because that shows just how how good they can be. So, the government has got a really difficult challenge, but it's a doable challenge.

[00:07:28]It has to work out what sort of tech stack we need, what sort of controls Like, people talk about this being a sort of freelance agent. Well, actually, to some extent that's not true. It was open AI's agent. They built it. It ran on their compute. How do you hold people liable for that, at what level? So, there are very difficult regulatory challenges, but they are manageable if we sit back, look at the details, and think about them strategically.

[00:07:51]>> Well, I I salute your calm attitude towards it. Kieran Martin, thanks very much for coming on.

[00:07:56]>> Thank you, Matt.