In February 2024 a researcher at JFrog went looking through the models people download for free, and found around 100 of them carrying live payloads.

Not proof of concept. Real code, with real destinations. One model opened a reverse shell to a hardcoded address the moment it loaded, handing whoever was on the other end command execution on the machine that ran it.

The platform's scanner had already looked at that file. It marked the model unsafe. Then it let people download it.

That is the whole problem in two sentences, and it is not really a story about one hub. It is a story about what a model file actually is.

A model file is a program

Most people picture model weights as a big spreadsheet of numbers. Inert. Data you load.

The most common format on public hubs does not work that way. PyTorch models are stored using Python's pickle, a mechanism for writing objects to disk and reconstructing them later. Reconstructing an object can require running code, so the format allows code, and loading the file runs it.

Python's own documentation does not hedge about this. The pickle module is not secure. Only unpickle data you trust. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling.

One clarification, because secondary coverage gets it backwards. The mechanism attackers abuse, `__reduce__`, controls how an object is written out, not how it is read back. The execution happens because reading a pickle means following instructions, and one of those instructions is "call this function".

That warning has been in the standard library docs for years. It was written for developers passing objects between their own processes. It now governs a supply chain in which enterprises pull weights from strangers on the internet, and the people doing the pulling are data scientists rather than release engineers.

Flagged, and downloadable

The detail worth sitting with is not that malicious models existed. Somebody will always upload something.

It is that the scanning worked on all 100 and the download still happened. The platform ran pickle scanning, identified files as unsafe, surfaced that label, and did not block or restrict the download.

Consider the equivalent anywhere else in your stack. Your email gateway detects a malicious attachment and tags the subject line rather than quarantining the message. Your endpoint agent identifies a trojan and writes a note in a log. Nobody would accept that, because detection without enforcement is a label, not a control.

The label also lands on the wrong person. It appears in front of someone who wants the model, is judging it on benchmark scores, and has no security mandate. Asking that person to overrule their own task is not a control either.

Then somebody beat the scanner

A year later, in February 2025, ReversingLabs found two models on the same hub that got past the scanning entirely. They named the technique nullifAI.

The method is elegant and worth understanding, because it explains why scanning this format is structurally hard. The models were stored in PyTorch format but compressed with 7z instead of the expected zip, so the standard loading function would not open them and the scanner did not recognize them as models to inspect.

The payload sat at the very beginning of the pickle stream. That placement is the trick. Pickle deserialization behaves like an interpreter, reading and acting on instructions as it encounters them, without first scanning the whole file. The malicious code ran, and only afterwards did the file hit its corrupted section and throw an error.

So the scan reported a broken file. The reverse shell had already connected.

Two models, one technique, 24 hours to removal after the 20 January notification. The hub removed them and improved the tool. That response was fast and correct, and it does not change the underlying property. A format that executes as it reads cannot be fully validated before it executes.

Why this reaches your organization

Ask who in your company has authority to add a Python package to a production system. There is probably a process, a list, a scan, an owner.

Now ask who has authority to download a set of model weights. Usually anybody, from anywhere, onto a machine that also holds credentials to your data.

The asymmetry is stark once you say it out loud. Adding one npm package to a build can require a ticket, a scan and a named approver. Pulling a 40 gigabyte file of executable weights from an account created last week requires a laptop and a network connection.

Models rarely appear in a software bill of materials. They do not go through dependency review. They are not versioned like libraries, and their contents cannot be read by a human the way source can. A data scientist evaluating six candidate models is running six programs from six strangers, and in most organizations that is not classified as installing anything.

The fix that already exists

There is a serialization format that solves this specific problem, and it is not exotic. Safetensors stores tensors and nothing else. No code, no execution path, nothing to run on load.

Two properties, both absolute. No code path, nothing to execute. Requiring it is a policy sentence, not a project. Models must be in safetensors, converted in a sandbox if the only available version is pickled, and pulled from a mirror your organization controls rather than directly from a public hub.

That last part matters more than it sounds. A mirror gives you a place to scan, a record of what was pulled and when, and the ability to remove something later. Pulling straight from the internet at training time gives you none of those.

What this is actually about

The deeper issue is that you cannot audit a model by reading it.

You can read source code. You can review a configuration. Weights are billions of numbers that mean nothing to inspection, so every judgment about a model is a judgment about its provenance. Who trained it, on what, and can you tell whether the file you have is the file they published.

That makes model supply chain a trust problem rather than an inspection problem, which is unfamiliar ground for security teams used to looking inside things. The controls that work are the boring ones. Restrict where files come from. Require formats that cannot execute. Keep a record. Verify hashes. Four controls, none of them new, all of them older than the problem.

None of that is glamorous, and none of it depends on predicting the next technique.

The test to run this week

Pick one model currently running in your environment. Any one.

Answer four questions. Where did the file come from. What format is it in. Who approved it. And if that file turned out to be malicious, on which machine did it first execute, and what else could that machine reach.

Most organizations can answer the first question and stall on the second. If you stall on the fourth, the exposure is not the model. It is that a download by one person became execution on a machine nobody was watching.

Then run the same four questions against your development machines rather than production. That is where the first execution happens, and it is where the credentials sit for the 12 or 20 systems a data scientist connects to in a week.

Then ask the question that follows. How many other model files are in your estate, and did anyone write them down?

Sources