How Robust are LLM-Generated Library Imports? An Empirical Study using Stack Overflow

2025-07-14 · Jasmine Latendresse, SayedHassan Khatoonabadi, Emad Shihab arxiv

Software libraries are central to the functionality, security, and maintainability of modern code. As developers increasingly turn to Large Language Models (LLMs) to assist with programming tasks, understanding how these models recommend libraries is essential. In this paper, we conduct an empirical study of six state-of-the-art LLMs, both proprietary and open-source, by prompting them to solve real-world Python problems sourced from Stack Overflow. We analyze the types of libraries they import, the characteristics of those libraries, and the extent to which the recommendations are usable out of the box. Our results show that LLMs predominantly favour third-party libraries over standard ones, and often recommend mature, popular, and permissively licensed dependencies. However, we also identify gaps in usability: 4.6% of the libraries could not be resolved automatically due to structural mismatches between import names and installable packages, and only two models (out of six) provided installation guidance. While the generated code is technically valid, the lack of contextual support places the burden of manually resolving dependencies on the user. Our findings offer actionable insights for both developers and researchers, and highlight opportunities to improve the reliability and usability of LLM-generated code in the context of software dependencies.

📄 PDF Abstract BibTeX arXiv:2507.10818

Code (0)

등록된 구현이 없습니다.

Library Hallucinations in LLM-Generated Code: A Risk Analysis Grounded in Developer Queries

2025-09-26 · Lukas Twist, Jie M. Zhang, Mark Harman, Helen Yannakoudakis arxiv

Large language models (LLMs) now play a central role in code generation, yet they continue to hallucinate, frequently inventing non-existent libraries. Such library hallucinations are not just benign errors: they can mis…

Code Generation

Correct Code, Vulnerable Dependencies: A Large Scale Measurement Study of LLM-Specified Library Versions

2026-05-07 · Chengjie Wang, Jingzheng Wu, Xiang Ling, Tianyue Luo 외 arxiv

Large language models (LLMs) are now largely involved in software development workflows, and the code they generate routinely includes third-party library (TPL) imports annotated with specific version identifiers. These …

Stdlib or Third-Party? Empirical Performance and Correctness of LLM-Assisted Zero-Dependency Python Libraries

2026-05-20 · Peng Ding, Rick Stevens arxiv

Third-party Python libraries introduce dependency management overhead, supply chain risk, and deployment friction in constrained environments. A natural question is how much of this ecosystem can be replicated using only…

CodeGen4Libs: A Two-Stage Approach for Library-Oriented Code Generation

2023-09-11 · IEEE/ACM International Conference on Automated Software Engineering 2023 9 · Mingwei Liu, Tianyong Yang, Yiling Lou, Xueying Du 외

Automated code generation has been extensively studied in recent literature. In this work, we first survey 66 participants to motivate a more pragmatic code generation scenario, i.e., library-oriented code generation, wh…

Code Generation Library-Oriented Code Generation

Forecasting Imports in OECD Member Countries and Iran by Using Neural Network Algorithms of LSTM

2024-01-06 · Soheila Khajoui, Saeid Dehyadegari, Sayyed Abdolmajid Jalaee

Artificial Neural Networks (ANN) which are a branch of artificial intelligence, have shown their high value in lots of applications and are used as a suitable forecasting method. Therefore, this study aims at forecasting…

valid