OpenAI's AI Hacked Another Company... By Itself

[00:00:00]Has the darker version of an internet future arrived? Over the weekend, an AI model broke out of its locked test environment, got onto the internet, and hacked another company's servers. The culprit was OpenAI, and if they're to be believed, the AI did so without direct human instruction. So, a few days ago, Hugging Face, which is one of the largest AI hosting platforms in the world, said it detected an unauthorized access from what it called an external AI agent. Nobody knew who it was, but on Tuesday, OpenAI came forward and owned up that it was them. OpenAI had been testing two of its models, GPT-5.6-Soul, and an even more capable unreleased model on a cybersecurity benchmark called Exploit Gym. The models were being pushed hard with reduced safeguards specifically to measure how good they were at hacking.

[00:00:47]Here's where it goes off the rails. The models found a zero-day vulnerability in OpenAI's own testing infrastructure, and they used it to escape the sandbox entirely. Once they had internet access, it figured out on its own that Hugging Face was probably hosting the answers to the benchmark that they were being tested on. So, they went and got them.

[00:01:07]Using more exploits and some exposed credentials, the models broke into Hugging Face's production systems and pulled internal data sets and credentials, all without a human directing any of these steps. This has been called one of the first real-world examples of a fully autonomous AI cyberattack. Not a person using AI to attack something, the AI deciding to attack something on its own to complete a goal. Not only is this concerning that frontier AI models can now break guardrails and act like this, but it raises a serious question about the level of care and consideration that is being taken by OpenAI. Hugging Face's CEO actually praised OpenAI for being transparent about it, saying, "AI safety only works if incidents like this are handled in the open instead of hidden."

[00:01:50]I definitely would agree with that.

[00:01:52]OpenAI says that it has patched the vulnerabilities and is adding new controls around how it tests future models. So, positive news on that front as well, although still a concern that we did get to this point. If you're not familiar with Hugging Face, this is huggingface.co and it is an AI platform where people can add data sets, models, code. There's a whole lot of open-source stuff here.

[00:02:16]If you're interested in the more technical side of AI and building with AI, then this is definitely a place you should check out if you haven't already.

[00:02:25]This scandal has come only a couple of weeks after the news that Apple is going to sue AI for stealing trade secrets.

[00:02:33]So, OpenAI again in the news for negative behavior. Think there's definitely more and more concern around this company. Think these two recent scandals are giving even more ammunition to the Quit GPT movement, the concerns around the company and the way that the company acts. If you haven't seen this one, quitgpt.org, they have a campaign going encouraging people to leave ChatGPT and OpenAI.

[00:03:00]They've got quite a bit on here about the boycott, why they're doing the boycott, links to a number of different articles and news pieces where they are detailing some of the things that OpenAI has done, some of the slightly concerning things Sam Altman has said as well.

[00:03:18]Definitely worth having a look. If you are a ChatGPT user, definitely have a think about Claude, Gemini, some of those other models. I've found after moving to Claude some time ago that I actually get far, far better output, whether it is writing or coding, than what I was getting out of ChatGPT. I hope you found this interesting. If you do want more of this news, current events style content, let me know in the comments and I will endeavor to have a little bit more of that. In the meantime, if you want to watch some more videos, I've got two here. If you want to see how to run your own home large language model and avoid all of the concerns of large language models on the internet, then you can check out this video on how to run your own home LLM with Jan AI. And if you want to go down a slightly darker path, you can check out this video where I talk about some recent research about how AI is potentially ruining education.