ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Data
Radiology report generation, as a key step in medical image analysis, is critical to the quantitative analysis of clinically informed decision-making levels. However, complex and diverse radiology reports with cross-source heterogeneity pose a huge generalizability challenge to the current methods under massive data volume, mainly because the style and normativity of radiology reports are obviously distinctive among institutions, body regions inspected and radiologists. Recently, the advent of large language models (LLM) offers great potential for recognizing signs of health conditions. To resolve the above problem, we collaborate with the Second Xiangya Hospital in China and propose ChatRadio-Valuer based on the LLM, a tailored model for automatic radiology report generation that learns generalizable representations and provides a basis pattern for model adaptation in sophisticated analysts' cases. Specifically, ChatRadio-Valuer is trained based on the radiology reports from a single institution by means of supervised fine-tuning, and then adapted to disease diagnosis tasks for human multi-system evaluation (i.e., chest, abdomen, muscle-skeleton, head, and maxillofacial $\&$ neck) from six different institutions in clinical-level events. The clinical dataset utilized in this study encompasses a remarkable total of \textbf{332,673} observations. From the comprehensive results on engineering indicators, clinical efficacy and deployment cost metrics, it can be shown that ChatRadio-Valuer consistently outperforms state-of-the-art models, especially ChatGPT (GPT-3.5-Turbo) and GPT-4 et al., in terms of the diseases diagnosis from radiology reports. ChatRadio-Valuer provides an effective avenue to boost model generalization performance and alleviate the annotation workload of experts to enable the promotion of clinical AI applications in radiology reports.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingLanguage ModelingLanguage ModellingLarge Language ModelMedical Image AnalysisMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
BioMistral-NLU: Towards More Generalizable Medical Language Understanding through Instruction Tuning
Large language models (LLMs) such as ChatGPT are fine-tuned on large and diverse instruction-following corpora, and can generalize to new tasks. However, those instruction-tuned LLMs often perform poorly in specialized m…
Instruction FollowingNatural Language UnderstandingQuestion AnsweringReading Comprehension+1Identifying and Extracting Rare Disease Phenotypes with Large Language Models
Rare diseases (RDs) are collectively common and affect 300 million people worldwide. Accurate phenotyping is critical for informing diagnosis and treatment, but RD phenotypes are often embedded in unstructured text and t…
Language ModellingLarge Language Modelnamed-entity-recognitionNamed Entity Recognition+4Utilisation d'indices phras\'eologiques pour \'evaluer des textes en langue \'etrang\`ere : comparaison des bigrammes et des trigrammes (Collocation measures and automated scoring of foreign language texts : Comparing bigrams and trigrams)
Cette recherche a pour principal objectif d{'}{\'e}valuer l{'}utilit{\'e} de prendre en compte des mesures totalement automatiques de la comp{\'e}tence phras{\'e}ologique pour estimer la qualit{\'e} de textes d{'}apprena…
SENTSCASS: Towards Building a Social-Support Chatbot for Online Health Community
Chatbots systems, despite their popularity in today's HCI and CSCW research, fall short for one of the two reasons: 1) many of the systems use a rule-based dialog flow, thus they can only respond to a limited number of p…
ChatbotGenerics in science communication: Misaligned interpretations across laypeople, scientists, and large language models
Scientists often use generics, that is, unquantified statements about whole categories of people or phenomena, when communicating research findings (e.g., "statins reduce cardiovascular events"). Large language models (L…