Model Retirement Creates Reproducibility Risk in Biomedical AI Publications
Background. Large language models (LLMs) are being adopted in biomedical research at a rapid and accelerating pace, yet commercial services that host many widely used models operate under deprecation schedules that can complicate scientific reproducibility. Methods. We searched PubMed for original research articles from 2022 through March 2026 that applied a specific LLM to a biomedical task. An extraction agent identified model names from 61,077 article abstracts with human reviewers validating a subset for extraction accuracy. Extracted model names were normalized to canonical model identifiers. Lifecycle data (release date, retirement date, status) were compiled for the 50 most frequently used models. Results. We identified 8,931 paper-model mentions spanning 5,242 unique publications after restricting the analysis to the 50 most frequently used models. Among these mentions, 77.7% cited a commercial closed-weight model. Overall, 42% involved a model that was already retired by the time of official publication or is scheduled to retire within two years of publication. The median interval from publication to model retirement was 538 days. Conclusion. Many biomedical publications using LLMs are on a trajectory toward computational non-reproducibility after publication. Model deprecation should be treated as a core reporting and preservation issue for biomedical research.
Code (2)
Similar Papers 제목 키워드 기반
Exploring Anti-Aging Literature via ConvexTopics and Large Language Models
The rapid expansion of biomedical publications creates challenges for organizing knowledge and detecting emerging trends, underscoring the need for scalable and interpretable methods. Common clustering and topic modeling…
Recommendations to enhance rigor and reproducibility in biomedical research
Computational methods have reshaped the landscape of modern biology. While the biomedical community is increasingly dependent on computational tools, the mechanisms ensuring open data, open software, and reproducibility …
Rescind: Countering Image Misconduct in Biomedical Publications with Vision-Language and State-Space Modeling
Scientific image manipulation in biomedical publications poses a growing threat to research integrity and reproducibility. Unlike natural image forensics, biomedical forgery detection is uniquely challenging due to domai…
Image ManipulationCommunity-developed checklists for publishing images and image analysis
Images document scientific discoveries and are prevalent in modern biomedical research. Microscopy imaging in particular is currently undergoing rapid technological advancements. However for scientists wishing to publish…
Enhancing Biomedical Knowledge Discovery for Diseases: An Open-Source Framework Applied on Rett Syndrome and Alzheimer's Disease
The ever-growing volume of biomedical publications creates a critical need for efficient knowledge discovery. In this context, we introduce an open-source end-to-end framework designed to construct knowledge around speci…
Benchmarking