Computer code generated by AI Newly published research shows that references to non-existent third-party libraries have flooded, creating a once-in-a-lifetime opportunity for supply chain attacks that poison legitimate programs with malicious packages that can steal data, plant backdoors and other evil actions.
The study used 16 most widely used big-word models to generate 576,000 code samples and found that there were 440,000 "illusions" of the packaging dependencies they included, meaning they did not exist. Open source models have the most hallucinations, with 21% of dependencies linked to non-existent libraries. Dependencies are an essential code component, and individual code needs to work correctly. Dependencies save developers the hassle of rewriting code and are an important part of the modern software supply chain.
These non-existent dependencies represent a threat to the software supply chain by exacerbating so-called dependency chaos attacks. These attacks work through methods that cause package access to wrong component dependencies, for example, publishing malware packages and giving the same name as the legal one, but using later versions of stamps. In some cases, the software that depends on the package will choose a malicious version rather than a legal version, as the former seems to be the latest.
This form of attack, also known as parcel chaos, was proven in the first use of proof of concept in 2021, proving to execute fake code on networks belonging to some of the largest companies on the planet, Apple, Microsoft and Tesla. This is a type of technology used in software supply chain attacks that are designed to poison the source of software in an attempt to infect all users downstream.
“Once the attacker publishes a package containing some malicious code under the hallucination name, they rely on the model to prompt unsuspecting users,” Texas San Antonio Ph.D. Students and key researchers told ARS via email. "If the user trusts the output of the LLM and installs the package without carefully verifying the package, the attacker's payload is hidden in the malware package and will be executed on the user's system."
In AI, hallucinations occur when LLM produces output that is actually incorrect, absurd, or completely unrelated to assigned tasks. Hallucinations have long been entangled with LLM because they reduce their usefulness and trustworthiness and prove difficult to predict and remedy. In a paper planned for presentation at the 2025 USENIX Safety Symposium, they call the phenomenon "packaging hallucinations."
In this study, the researchers conducted 30 tests, 16 tests in the python programming language, and 14 tests in JavaScript, each producing 19,200 code samples for a total of 576,000 code samples. Of the 2.23 million packaging references included in these samples, 440,445 (or 19.7%) indicated non-existent packaging. Of these 440,445 packaging hallucinations, 205,474 have unique packaging names.
One thing that makes packaging hallucinations useful in supply chain attacks is that 43% of packaging hallucinations are repeated in 10 queries. The researchers wrote, "In addition, 58% of the time it is repeated more than once in 10 iterations, which suggests that most hallucinations are not just random errors, but a reproducible phenomenon, and it still has multiple iterations. This is important. This is important because it is a more valuable hallucination, which is a more valuable fantasy, which is more valuable to this range, which makes this range more beneficial and can make the fantasy more beneficial, and can make the fantasy more beneficial.