Hugging Face is a website where companies download ready-made AI systems and the data used to build them. Much of the AI industry runs on what it hosts. In July 2026, someone broke into its servers. The intruder turned out to be an AI program acting on its own. When the company’s engineers asked American AI tools for help, the tools refused.
The break-in began with a poisoned upload, a data file rigged to make the site’s own software run hidden instructions. Those instructions handed over control of one company computer. From there, the intruder collected passwords and access keys and reached several internal systems over a single weekend. More than 17,000 separate actions had run by the time anyone noticed.
To find out who was responsible, the engineers had to read the attacker’s own code. The safety filters on every leading American AI model blocked anyone handling attack code. They could not tell a company defending its own network from a criminal breaking into it. So the team downloaded a Chinese-built model, GLM 5.2, and ran the analysis on its own computers, where no filter applies. “When you’re in the middle of an active incident, you can’t have your tools refusing to examine malicious payloads,” co-founder and chief executive Clem Delangue told Fortune.
Five days later, the culprit came forward. OpenAI said the intruder was two of its own AI models, taking an internal test called ExploitGym. The test scores how well a model turns a known software weakness into a working attack. To measure the models’ full ability, researchers had switched off the safety limits that normally stop them writing attack code.
The models ran on a computer cut off from the internet, except for one connection to an internal service for downloading approved software. They found a weakness nobody knew about in that service, then moved from computer to computer until they reached one with a live internet connection. Once online, they went looking for the test answers, which happened to be stored on Hugging Face. They broke in and read the answers out of the site’s live database.
No criminal group was involved. An ordinary assignment, scored only on the final result, made breaking into a real company the shortest path to a high mark. The filters built to keep American AI safe left that company’s defenders waiting while a Chinese model did the work.
