Hugging Face Breach: Hackers Exploit Vulnerability, Steal Internal Data and Credentials
Hugging Face, a leading platform for hosting and sharing AI models and datasets, has confirmed a breach that compromised its internal datasets and service credentials. The company disclosed the incident last week, but emphasized that it is still investigating whether any customer or partner data was stolen during the attack.
The breach occurred when a dataset uploaded to the Hugging Face platform exploited a security vulnerability, allowing the attackers to run malicious code on the company’s servers. This enabled them to escalate their permissions and gain broader access to Hugging Face’s internal systems.
In a blog post, Hugging Face revealed that the attackers used an external AI agent to execute ‘many thousands of individual actions across a swarm of short-lived sandboxes’ with self-migrating command-and-control staged on public services. The company credited its own anomaly detection system for spotting the attack, which used an AI model to analyze server logs and keep a record of the cyberattack.
Hugging Face initially employed a frontier AI model from a commercial provider to analyze the attack, but found that the analysis effort was blocked by the provider’s guardrails. The company then turned to its own local large language model, which provided the added benefit of not having to upload sensitive attack logs to an AI company’s servers.
The incident highlights the challenges that companies like Hugging Face face when hackers try to abuse platforms and tools to access and steal sensitive data from within. Security researchers have previously complained that some frontier models, like Anthropic’s Mythos and Fable, are heavily constrained and prevent defenders from inquiring about almost anything relating to cybersecurity.
Hugging Face has reported the incident to law enforcement and roped in cybersecurity forensic specialists to investigate the breach and review its security. The company has also urged users to take action and review any suspicious activity on their accounts, as well as to do the same with any keys stored on the platform.
Hugging Face’s breach serves as a stark reminder of the importance of robust security measures and the need for companies to stay vigilant against emerging threats in the AI and cybersecurity landscape.
As the investigation into the breach continues, Hugging Face remains tight-lipped about the potential implications for its users and partners. The company has not provided any evidence to support its claim that an external AI agent was responsible for the attack, and has not commented on whether it had performed a security audit of its systems before launching.
While the full extent of the breach is still unclear, one thing is certain: the incident has sent shockwaves through the AI and cybersecurity communities, and serves as a stark reminder of the need for companies to prioritize security and vigilance in the face of emerging threats.
Hugging Face’s breach also raises questions about the use of frontier AI models in cybersecurity and the potential risks associated with these models. As the company continues to investigate the incident, it is likely that the AI and cybersecurity communities will be watching closely to see how Hugging Face responds to this challenge.
