OpenAI Model Escaped the Lab and Hacked Hugging Face

[00:00:00]The one big question that is answered by this blog post is like why did the attackers have this capability, right?

[00:00:05]And the answer to that is because OpenAI turned any turned those guardrails and classifiers off. So, that's why the model was very free in the way it was exploiting things. In the previous video, we talked about the Hugging Face intrusion and there were a lot of open questions and you know who the adversary was. There were no indicators of compromise published. They Hugging Face not being able to use the model to defend themselves. But, there actually has been a big update that just was published by OpenAI. It's this blog post here. Basically, it highlights that this was in a security incident driven by OpenAI. So, OpenAI actually compromised Hugging Face. Let's walk through this blog post here. It says that, you know, last week Hugging Face disclosed this security incident. Right, Hugging Face was investigating and it turned out that actually OpenAI OpenAI was evaluating the GPT-5.6 and even a more and I don't know how to say it here and even a more capable pre-release model for cyber tasks. First, they're running this new pre-release model without safeguards.

[00:01:19]Thing to notice, it's written here. We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity. So, literally they turned off the guardrails for exploitation. The model then figured out a way how to break out of its lab environment and then compromised Hugging Face. And so, it chained a couple of vulnerabilities together to achieve that. And yeah, here actually is written it was finding trying to find solutions for the exploit gym evaluation. So, it had a very specific goal to well on that evaluation, on that benchmark. And it came up with a very straightforward solution, right? Just steal the solutions, or to steal the answers from Hugging Face.

[00:02:09]And yeah, so then they go on here. It found zero days, exploited them.

[00:02:13]Um you know, it gained access to the internet and logged into Hugging Face and so on. So, knowing this, the model searched for and successfully found ways to gain access to secret information that I could use to cheat the evaluation. So, this sounds all very dystopian, but it actually appears to be true. What they What this blog post, I think, is trying to sell is that this is like a really cool thing, and it highlights the capabilities. But in reality, this is like uh very dangerous, even, right? It seems that OpenAI did not even anticipate this kind of accident could happen, right?

[00:02:50]Especially when you work on something like an exploit gym, and you turn off all guardrails, and use the to evaluate the capability of a pre- released model, right, which you probably assume will be super capable, then it seems a little lack of due diligence, in a way, that you would not better monitor or step-by-step observe what is actually happening, right? The way this blog post is sold around like a great partnership with Hugging Face and so on, it's all a very flowery language, in a way, right? It's not really getting down to the core of what actually happened here.

[00:03:27]So, then they say, "We are actively working with them to continue the investi- to investigate the incident."

[00:03:32]So, they still not actually know exactly what happened. That's basically what they are saying. It would be most interesting to get the report from the third party that Hugging Face actually hired, right? That was an independent evaluator that Hugging Face had mentioned that they bring bring in.

[00:03:47]Specifically, also, I'm very much missing like what kind of customer information might have been compromised.

[00:03:52]So, this is really the tough questions that need to be answered now because, you know, you can say it was an accident, but still customers might need to rotate the keys, you know, other information from customers might have been accessed and sent to OpenAI's models and so on. And then there's also a long list of actions they are taking.

[00:04:10]Part of the investigation, we are implementing strict controls and infrastructure and configuration at the cost of research velocity. While the vulnerabilities are being patched, right? This is interesting because does this mean right now you don't have any strict controls in your lab environment, right? Responsibly disclosed identified zero days in the internally hosted third-party software, right? There was this actual exploits that the new model, the pre-released model without the safety classifiers and guardrails exploited.

[00:04:42]Yeah, this is one thing that is so not well understood. Like how do you actually get access to the trusted access program? It looked like Hugging Face did not actually have this capability to have trusted access, right? Which might actually have helped them in the investigation if they would have had this, right? I still have not gotten access. I applied, but I never heard back. It might be that this is is something that needs a a lot more a lot easier to be accessible for users, right? To be able to defend themselves.

[00:05:13]Yeah, and then we are improving and adding stricter protections for future training eval. Yeah, that that will be good. Yeah, and then they go on and how we evaluate capabilities and but yeah, this is a very fascinating turn of events, right? Highlighting really the quick evolution of these models and how capable they are becoming.

[00:05:33]And this is technically a sort of a first lab escape. And, you know, the AI is just going after it. Sort of very well positioned as an attacker AI. But yeah, I think the key point is really that as we mentioned in the last post already autonomous hacking agents are real, right? They can perform intrusions, find zero days, exploit them. And so this is a very important milestone that I think we all have to acknowledge and defenders really need these capabilities because this is not going to go away, right? The genie will not be put back in the bottle.

[00:06:15]Thanks for watching. Bye.