Unlocking Historical Clinical Trial Data with ALIGN: A Compositional Large Language Model System for Medical Coding
The reuse of historical clinical trial data has significant potential to accelerate medical research and drug development. However, interoperability challenges, particularly with missing medical codes, hinders effective data integration across studies. While Large Language Models (LLMs) offer a promising solution for automated coding without labeled data, current approaches face challenges on complex coding tasks. We introduce ALIGN, a novel compositional LLM-based system for automated, zero-shot medical coding. ALIGN follows a three-step process: (1) diverse candidate code generation; (2) self-evaluation of codes and (3) confidence scoring and uncertainty estimation enabling human deferral to ensure reliability. We evaluate ALIGN on harmonizing medication terms into Anatomical Therapeutic Chemical (ATC) and medical history terms into Medical Dictionary for Regulatory Activities (MedDRA) codes extracted from 22 immunology trials. ALIGN outperformed the LLM baselines, while also providing capabilities for trustworthy deployment. For MedDRA coding, ALIGN achieved high accuracy across all levels, matching RAG and excelling at the most specific levels (87-90% for HLGT). For ATC coding, ALIGN demonstrated superior performance, particularly at lower hierarchy levels (ATC Level 4), with 72-73% overall accuracy and 86-89% accuracy for common medications, outperforming baselines by 7-22%. ALIGN's uncertainty-based deferral improved accuracy by 17% to 90% accuracy with 30% deferral, notably enhancing performance on uncommon medications. ALIGN achieves this cost-efficiently at \$0.0007 and \$0.02 per code for GPT-4o-mini and GPT-4o, reducing barriers to clinical adoption. ALIGN advances automated medical coding for clinical trial data, contributing to enhanced data interoperability and reusability, positioning it as a promising tool to improve clinical research and accelerate drug development.
Code (0)
등록된 구현이 없습니다.
Tasks
Code GenerationData IntegrationLanguage ModelingLanguage ModellingLarge Language ModelRAGMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Leveraging Historical Data for High-Dimensional Regression Adjustment, a Composite Covariate Approach
The amount of data collected from patients involved in clinical trials is continuously growing. All patient characteristics are potential covariates that could be used to improve clinical trial analysis and power. Howeve…
regressionSECRET: Semi-supervised Clinical Trial Document Similarity Search
Clinical trials are vital for evaluation of safety and efficacy of new treatments. However, clinical trials are resource-intensive, time-consuming and expensive to conduct, where errors in trial design, reduced efficacy,…
A Machine Learning Approach for Recruitment Prediction in Clinical Trial Design
Significant advancements have been made in recent years to optimize patient recruitment for clinical trials, however, improved methods for patient recruitment prediction are needed to support trial site selection and to …
BIG-bench Machine LearningTrial2Vec: Zero-Shot Clinical Trial Document Similarity Search using Self-Supervision
Clinical trials are essential for drug development but are extremely expensive and time-consuming to conduct. It is beneficial to study similar historical trials when designing a clinical trial. However, lengthy trial do…
Clinical KnowledgeRetrievalIncreasing the efficiency of randomized trial estimates via linear adjustment for a prognostic score
Estimating causal effects from randomized experiments is central to clinical research. Reducing the statistical uncertainty in these analyses is an important objective for statisticians. Registries, prior trials, and hea…