Mon, 10 Aug 2026 UTC

# OpenAI / Hugging Face hack

I think this Hugging Face / OpenAI “model sandbox escape” drama is overhyped but not for the reasons I’ve seen written up in many places. And I think that we are still stuck on OpenAI’s preferred framing, which is that their model has scary capabilities that allowed it to escape a secure sandbox.

Yes, this hack was objectively impressive. The exploit development, pivoting, and privilege escalation was quality work. See Simon Willison’s write up for details.

However there’s a dirty secret that anyone who has ever worked in information security will have quickly internalised: software quality is shit. This has been true throughout history, and is still true in 2026. Open source software quality is shit. Proprietary software quality is worse. There are vanishingly few pieces of software that are reasonably good quality: I’m thinking projects like curl, SQLite, or Go. And those are only good because the people behind them put an enormous amount of energy and focus into keeping the shittiness at bay. And even those projects still regularly have security vulnerabilities.

Allowing an AI model to grind away against Artifactory for weeks was always going to result in exploitation. It is all but guaranteed that there are exploitable bugs in this kind of software or in its configuration. If OpenAI considers a model with access to an Artifactory proxy that has onwards internet access to be in a secure sandbox, then they do not understand software or information security.

It is akin to a builder leaving a skip full of demolition waste on the verge and then being shocked when the wind comes up and blows asbestos dust into the neighbour’s backyard. And the response from regulators should be the same: keep your shit secured, or face legal penalties.

A full skip. Credit: Snowmanradio CC BY-SA 3.0

A full skip. Credit: Snowmanradio CC BY-SA 3.0

So don’t allow OpenAI to frame this as the first warning shots fired in the war against the machines. Instead see it as evidence that for all the money and resources they are sucking up, OpenAI are still subject to the same fundamental nature of software as the rest of us. For what it’s worth I believe that people at OpenAI know all this. They just have fantastic marketing and currently operate in a regulatory vacuum.

tags : hack ai opinion