We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs

A comprehensive analysis of package hallucinations by code-generating LLMs. The companion cites its finding that code models invent names of nonexistent software packages, at rates of at least 5% for commercial models and 22% for open ones, and that attackers can register them. It is a niche built by the errors of one kind of organism and occupied by another.

Cited in

the companion paper (Companion §4)

Used for

Niches; food web (§4)

Strand

Cybersecurity & AI-enabled attack · Prompt injection, agent worms & supply-chain parasites

Cite

@inproceedings{spracklen2025package,
  title = {We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs},
  author = {Joseph Spracklen and Raveen Wijewickrama and A H M Nazmus Sakib and Anindya Maiti and Bimal Viswanath and Murtuza Jadliwala},
  year = {2025},
  howpublished = {Proceedings of the 34th USENIX Security Symposium},
  url = {https://arxiv.org/abs/2406.10279},
}