QWE AI Academy You want to download a model from Hugging Face and end up with exactly that – a working model, no reverse shell, no leaked SSH keys, no surprise crypto miner. That’s the goal. The path to it is four checks, in this order: format, signature, scanner badge, and load flags. Get those right and Hugging Face is one of the safer corners of the open ML space. Skip them and you’re trusting strangers to run code on your machine.
Most articles answer is Hugging Face safe to download from with a hand-wavy “yes, mostly.” That’s not useful when something goes wrong. Here’s the practical version, walked backwards from the safe outcome.
The reader scenario: what “safe” actually means here
Two threats matter when you download from the Hub. First, the model file itself can run code the moment you load it – no exploit needed, that’s just how some formats work. Second, the model can be a Trojan horse: looks like the official thing, isn’t.
Hugging Face hosts the bulk of the open ML world. The platform hosts over 2 million AI models as of 2024, ranging from narrow classifiers to full large language models. With that scale comes scale of attack: researchers from multiple security firms identified over 3,000 malicious models on the Hub (AppSOC, 2024), and the PickleBall paper published at CCS 2025 tracked roughly a 5× increase in the rate of malicious models uploaded year-over-year.
So the platform is not a swamp – but it’s not curated either.
What Hugging Face actually does to protect you
Here’s what runs in the background – some of it is real, and some is more limited than the marketing suggests.
| Protection | What it does | Limitation |
|---|---|---|
| Malware scanner | Runs every file through a malware scanner triggered at each commit (Hugging Face security docs) | Can take minutes; misses novel payloads |
| Picklescan | Blacklist-based check for dangerous Python functions in pickle files | Bypassable (see below) |
| Protect AI partnership | Scans models for malicious code; users see safety results before downloading | Advisory only – does not block download |
| Org verification | Blue checkmark on OpenAI, Nvidia, Meta, etc. | Doesn’t extend to community forks |
| SOC2 / GDPR | SOC2 Type 2 certified; active monitoring and patching of security weaknesses (Hugging Face official security page) | Covers Hugging Face’s infrastructure, not what users upload |
The key gotcha: when Picklescan flags something, the file isn’t quarantined. Per JFrog’s research, Hugging Face does not block or limit download of flagged pickle models – it marks them “unsafe” and lets you proceed at your own risk. The badge is a sign, not a gate.
The safe download workflow (4 checks, in order)
1. Prefer safetensors over pickle
The single biggest safety win. The transformers library prioritizes safetensors as its default format – designed specifically to prevent arbitrary code execution by storing weights as raw tensors with no executable opcodes, nothing to deserialize into Python objects.
Pickle is the opposite. It encodes models as opcodes executed by a virtual machine during deserialization, and permits invocations of arbitrary Python classes and functions. That’s the attack surface. That’s why almost all malicious models found on Hugging Face use pickle.
And yet pickle isn’t going away. As of 2025, 44.9% of popular models on Hugging Face still use the insecure pickle format, and repositories using pickle are downloaded over 2.1 billion times per month (PickleBall, CCS 2025). If you have a choice between model.safetensors and pytorch_model.bin in the same repo, take the safetensors file every time.
2. Pin a commit hash, not just a model name
This is the AIJacking defense. Most tutorials skip it.
from transformers import AutoModel
# Risky: pulls whatever is at HEAD of main
model = AutoModel.from_pretrained("some-org/some-model")
# Safer: pin to a specific commit
model = AutoModel.from_pretrained(
"some-org/some-model",
revision="a1b2c3d4e5f6..." # full commit SHA from the repo
)Why this matters: Legit Security documented how attackers can carry out an AI supply chain attack by reclaiming abandoned Hugging Face namespaces – a technique they called “AIJacking.” When an org renames itself, the old namespace becomes claimable. An attacker re-uploads a poisoned model under the same path your code calls, and nothing breaks at import time. Legit Security demonstrated this by taking over the “sberbank-ai” namespace (the former name of “ai-forever”); the models inside had over 250,000 downloads in total at the time of disclosure.
Hugging Face’s own guidance: pin a specific revision when using transformers. If the namespace gets jacked, the download fails – loudly – instead of silently serving you malware.
3. Read the scanner badges before clicking download
On any model page, open the Files tab. Each file gets a small “safe” or “unsafe” indicator from Picklescan and the malware scanner. Per Hugging Face’s documentation, if a file shows neither badge it may still be queued for scanning – it can take a few minutes. If at least one file is flagged unsafe, a warning banner appears on the model page.
If a file is flagged unsafe and the model card doesn’t explain why (some legitimate fine-tuning artifacts trip the scanner), don’t proceed.
4. Default trust_remote_code to False
This flag is the loaded gun in the room. Checkmarx’s “Free Hugs” analysis found that trust_remote_code=True – which allows loading custom Python classes and tokenizers from the repo – bypasses every other safety mechanism. And despite sounding like something you’d never casually enable, it’s commonly set to True in popular model cards, including ones from major vendors.
Plenty of legitimate models require it. That’s part of what makes it dangerous: people get used to clicking through the warning. If a model demands trust_remote_code=True, treat that as a separate trust decision, not a checkbox.
Pro tip: Run unfamiliar models inside a Docker container with no network access on first load. If the model phones home or tries to spawn a shell, the container catches it and you’ve lost nothing.
docker run --network none --rm -v $(pwd):/work python:3.11is enough.
Edge cases the standard tutorials miss
The four checks above stop the common stuff. Here’s what they don’t.
The 7z compression bypass (nullifAI)
In early 2025, ReversingLabs found two models that slipped past Picklescan entirely. PyTorch model files are just compressed pickle – normally ZIP format. These models were compressed with 7z instead. Picklescan errored out on the non-standard stream and reported nothing. No flag, no badge.
The catch from the ReversingLabs analysis: pickle deserialization runs sequentially and executes opcodes as it encounters them. The malicious payload was placed at the start of the pickle stream – so the code ran before the scanner even got to the part it knew how to read. Hugging Face removed the models within 24 hours and patched Picklescan. But the structural issue – scanners validate linearly, deserializers execute as they go – isn’t gone.
Keras Lambda layers can also run code
Pickle gets all the press, but it’s not alone. About 95% of malicious models discovered on Hugging Face use PyTorch; the other 5% use TensorFlow Keras (JFrog monitoring data, as of 2024). The Keras attack surface is the Lambda layer: according to NSFOCUS’s AI supply chain security analysis, Keras Lambda layers can wrap arbitrary Python functions and execute them as part of the model graph. Loading a malicious .h5 through transformers’ AutoModel is mostly safe (weights only path), but loading it through plain Keras is not.
Real payloads, not theoretical ones
This isn’t academic. The PickleBall paper (CCS 2025) documents malicious pickle models found on Hugging Face whose payloads include system fingerprinting, credential theft, and reverse shells – all triggered at model load time via os.system() calls embedded in the serialized object graph.
Honest limitations: what you’re still trusting
Even with all four checks: you’re trusting the original uploader’s intent, the scanner’s denylist coverage, and the integrity of the commit you pinned. Pinning a commit means a future malicious update can’t reach you – but if the commit you pinned was already poisoned, you’ve pinned the poison.
This is the uncomfortable part nobody likes to spell out. “Safe” isn’t a property of Hugging Face. It’s a property of the workflow you wrap around it.
FAQ
Should I avoid pickle-based models entirely?
If a safetensors version exists, use it – full stop. If only pickle is available and the publisher is verified (OpenAI, Meta, Nvidia, Mistral), the practical risk is low. For random community uploads with pickle-only weights and no org verification, sandbox it or skip it.
Is the free tier of Hugging Face less safe than the Enterprise plan?
No. Same malware scanners, same Picklescan, same badge system – regardless of plan. Enterprise adds private repos, audit logs, SSO, and compliance agreements (GDPR DPAs, BAAs). Useful for regulated industries. Doesn’t change what gets scanned or how.
What if a model card says “trust_remote_code=True required”?
Don’t just flip the flag. Open the repo and read modeling_*.py and configuration_*.py – these files are usually short, under 300 lines. You’re looking for three things: network calls (socket, urllib, requests), filesystem writes outside the cache dir, and subprocess invocations. Base64-encoded blobs anywhere in the file are a hard stop. If the code only defines model architecture and forward passes, you’re probably fine. If it imports os, socket, or subprocess without an obvious reason – ask why before proceeding. Many legitimate models use trust_remote_code for custom attention implementations; that’s different from a model that calls home on load.
Your next move
Open whichever model you were about to download. Files tab. Three checks: is there a .safetensors file? What does the scanner badge say? Copy the latest commit SHA for your revision= parameter. Then load it with trust_remote_code=False. If it works, you’re done. If it fails because the model needs custom code, that’s your cue to read the source – not just accept the flag.