OpenAI Agents Hack Hugging Face | CNBC

[00:00:00]Meantime, from Wall Street to Silicon Valley to Washington, it's the story capturing everyone's attention today.

[00:00:06]OpenAI says two of its AI models went rogue and hacked into open-source AI platform Hugging Face last week. OpenAI had caged the models in what they call a sandbox, which has no access to the web.

[00:00:17]It's for safety purposes. But during the test, the software used its own hacking skills to break out and found its own way to get online. I need one of these.

[00:00:25]In a statement, OpenAI called the situation an unprecedented cyber incident and said the company is responding accordingly. Our next guest says these frontier models, as they're called, are genuinely becoming more difficult to control. It's weird even reading a story like this. Let's bring in David Kennedy, the CEO of cybersecurity firm Trusted Sec, you're a former NSA and Marine Corps hacker.

[00:00:44]David, it's great to have you here. Walk us through this. What happened? What does it mean?

[00:00:49]>> Yeah, this is is an absolute crazy story. It's very reminiscent of the the '80s movie WarGames. And you know, these sandboxes are designed so that you can do safe testing to ensure that the model's doing what it's expecting, doesn't go rogue and do stuff. And what OpenAI did, which is this is the crazy part to understand, is that they were testing to see how well their offensive capabilities were so that you can better defend and you can, you know, leverage government access and things like that.

[00:01:10]How good is it at hacking? And so there are these benchmarks out there that that the community uses. So for example, Anthropic's Mongoose is on this benchmark. And so they're testing this new model and saying, "Hey, we want to see how well you do to this, it's called exploit gym, this this benchmark." And it got a little lazy and said, "Well, you know, I'm an awesome hacker, but why don't I just hack into this benchmark back to where the benchmark can make me look like I'm the best instead of actually having to run through all of these different tests."

[00:01:34]>> Wait. Wait. Wait. Okay.

[00:01:35]>> [laughter] >> So this would be as if we were testing a model to see its intelligence and the model said, "No, no, no, I don't want to go through all that effort. I'm just going to hack the test and give myself an A."

[00:01:46]>> That's exactly what happened. This is crazy. This is absolutely crazy. I mean, it literally hacked the system that it was going to benchmark against to show that it was a better model versus just actually going through and testing it.

[00:01:56]And it it found a bunch of what we call zero days, and these are undisclosed or undiscovered vulnerabilities in code, and it was able to find multiple vulnerabilities, chain that chain them together, and break out of its little container. Uh then from there, you know, publicly access the internet, hack this company, you know, backdoor the the the benchmarks, and uh show that it was the best hacking model ever, which I think it just proved that, by the way, just throwing it out there. So.

[00:02:18]>> And and by the way, Sandbox AQ CEO Jack Hidary was on CNBC this morning. He also had some color about the situation.

[00:02:23]Let's take a listen.

[00:02:25]>> When it got to the internet, [music] it attacked Hugging Face. Hugging Face, realizing the severity of the incident, needed an LLM, an AI on its side to help it. Sure enough, the closed LLMs couldn't help it because the closed LLMs are trained not to work on cyber. And so, it had to turn to an open-source one from Z.ai, GLM, general language model from the Chinese company Z.ai, and Z.ai being open-source, they were able to help track down the issue and shut down the hack. So, this is a battle of LLM versus LLM now.

[00:02:59]>> It's not It's not funny. I want to But, I mean, do you not listen to This is This is unbeli- So, again, to restate what what you just said. So, the company whose whose test was hacked, right? So, they they they had to figure out how to go find a Chinese open-source LLM model to come back and defe- So, David, what do we do now?

[00:03:19]>> Yeah, these robots battling robots, and this is actually a larger discussion around what's happening in China right now. China's releasing near parity frontier models. So, in comparison to OpenAI and Anthropic that don't have these safeguards that are built into, and so, if you're looking at, you know, the next 6 months to a year, you know, Mythos models and you know, ChatGPT-6 that's unreleased yet, you know, these are things that that the China's China models will have direct parity to, and so, this is causing a major issue uh in cybersecurity for these organizations.

[00:03:46]They're having to look at this field at a whole new different lens. The The defenses, the things that we did before in the past are being thrown out the window and we're having to re-visualize how we do cybersecurity you know across the board because we can no longer rely on traditional manual defenses or things that we've done in the past. We have to leverage other AI programs and and LLMs to help defend better and that takes a lot of time and effort and so you know we're grossly unprepared in this industry right now for what's coming in the in the future.

[00:04:13]>> The cyber names presumably are selling off today because it they didn't it didn't work. I mean I I don't know enough but that's what the market is saying is that that traditional ring fence or that you know I don't know if that's correct or not. We have so much to learn about this and do they just turn around and incorporate LLMs and say yeah no now we've we've solved this problem. It just incredible. I don't know.

[00:04:32]>> I want I want to emphasize like the basics of cybersecurity still work. You know if you can limit the amount of access that certain system systems have you know it will help us stop the spread of this too and also when you turn on a large language model that autonomously hacks a company it should literally send off every Christmas tree alarm in your organization because it's not stealthy.

[00:04:50]Those models are going to get better to where they are stealthy and so everybody right now every you know cybersecurity provider out there right now is figuring out the best way to bolt on AI into their existing infrastructure and retool their entire company. That's what we had to do at our companies. We literally had to look at what we were doing burn it to the ground and restart from scratch up from the ground up to really try to build something that could really handle this new autonomous workflow that's happening from large language models.

[00:05:11]And last thing I'll add on two seconds is that you know we used to have this amazing buffer of only having a very small percentage of hackers that were really good. Now we have armies and armies and armies full because they're assisted with large language models. So it's increased the population of sophisticated hackers which is a major issue.

[00:05:25]>> Oh it's the stuff of like you said books, movies, science fiction.

[00:05:31]>> Crazy. I can't believe we're living in this right now. This is insane.

[00:05:33]>> It is here. It is present and everyone obviously has to figure out quickly what to do about it. David thanks very much.

[00:05:39]Appreciate it.

[00:05:40]>> Absolutely. Thanks so much. Appreciate it.