MilikMilik

How Malicious AI Models and Repos Weaponize the Open-Source Supply Chain

How Malicious AI Models and Repos Weaponize the Open-Source Supply Chain
Interest|High-Quality Software

AI Supply Chain Attacks: When Trust Becomes the Attack Surface

AI supply chain attacks are incidents where attackers abuse trusted AI platforms, pre-trained models, or open-source code repositories to deliver malware or steal data by hiding malicious behavior inside assets that developers routinely download and integrate into their applications. As teams depend on third-party models and code to speed up development, attackers increasingly target these dependencies instead of direct network perimeters. A poisoned model or compromised repository can reach thousands of systems through a single from_pretrained() call or package install, turning developer workflows into distribution channels for malware delivery. The core problem is misplaced trust: tools and libraries assume that configurations, models, and example code are safe, while developers often run them with powerful permissions and access to secrets. This combination of implicit trust and deep integration makes AI model security a critical part of modern supply chain defenses.

The Hugging Face Transformers Flaw: Code Execution from a Model Config

The Pluto researchers’ discovery of CVE-2026-4372 in Hugging Face Transformers shows how a small configuration field can hide a major code execution vulnerability. The bug appears when the optional kernels package is installed, which is common in GPU-accelerated environments and popular installation options. According to eSecurityPlanet, one poisoned field in a model’s config.json can “silently execute arbitrary code on anyone who loads it” through a standard from_pretrained() call, even when trust_remote_code=False is set. The root cause is a generic setattr() mechanism that writes configuration values directly into internal objects, including private attributes. By changing the _attn_implementation_internal setting to reference a malicious kernel repository, attackers can trigger automatic download and import of attacker-controlled Python code, leading to remote code execution and potential credential theft. Vulnerable versions were downloaded about 232 million times, underscoring the scale of AI supply chain exposure.

Compromised GitHub Repositories Targeting Claude and Gemini Users

Open-source repositories are another weak point in AI supply chains. When attackers compromise a GitHub repo that targets users of popular AI tools, they gain a direct path into developer environments. In a recent incident, Microsoft shut down more than 70 GitHub repositories after discovering they were being used to deliver malware to people working with AI assistants such as Claude and Gemini. Although source details are limited, the pattern is clear: attackers seed or modify code that appears to help integrate or automate AI workflows, then hide payloads that activate when developers run setup scripts or sample automation code. Because many teams trust code hosted under familiar accounts or ecosystems, they may skip detailed review, granting the malicious code access to local files, environment variables, and authentication tokens. This expands the blast radius of a compromised repo from a single developer to entire connected systems.

How Malicious AI Models and Repos Weaponize the Open-Source Supply Chain

Credential Theft Through AI Agents and Model Loading

AI tools now run close to critical secrets. Coding agents, data pipelines, and inference services often hold cloud credentials, SSH keys, and API tokens so they can deploy, test, or manage resources. A successful AI supply chain attack turns this convenience into a direct path to credential theft. The Transformers vulnerability showed that code execution during model loading can expose cloud credentials, API tokens, and SSH keys, giving attackers a foothold in enterprise infrastructure. Similarly, compromised repositories that set up AI coding agents can exfiltrate configuration files, environment variables, and local key stores. Once stolen, these credentials allow attackers to move laterally across CI/CD systems, source control, and production clusters. The risk is not limited to a single infected workstation; it extends to every system that trusts those keys, making AI model security and agent isolation essential for protecting sensitive data access.

Improving AI Model Security: Verification, Isolation, and Least Privilege

Current defenses often treat AI components as ordinary libraries, which is no longer enough against targeted supply chain attacks. Security teams need explicit controls tailored to AI model security. For platforms like Transformers, upgrading to patched versions and auditing environments that include the kernels package are immediate steps. More broadly, organizations should maintain an AI-focused software bill of materials and asset inventory, so every external model and dependency is tracked. Before using third-party models or AI agent code in production, run them in isolated, sandboxed environments with restricted outbound network access and minimal credentials. Apply least-privilege principles by removing long-lived keys from model-loading hosts and limiting what AI tools can access. Monitoring should cover unusual model downloads, repository references, and package imports tied to AI workloads. Combined with tested incident response plans, these measures help contain malware delivery and supply chain attacks before they reach core systems.

Milik earns a commission when you shop through our links, at no extra cost to you. Editorial content is independently selected by our team.

You May Also Like

Comments
Say something...
No comments yet. Be the first to share your thoughts!