Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports
Neural abstractive summarization models are able to generate summaries which have high overlap with human references. However, existing models are not optimized for factual correctness, a critical metric in real-world applications. In this work, we develop a general framework where we evaluate the factual correctness of a generated summary by fact-checking it automatically against its reference using an information extraction module. We further propose a training strategy which optimizes a neural summarization model with a factual correctness reward via reinforcement learning. We apply the proposed method to the summarization of radiology reports, where factual correctness is a key requirement. On two separate datasets collected from hospitals, we show via both automatic and human evaluation that the proposed approach substantially improves the factual correctness and overall quality of outputs over a competitive neural summarization system, producing radiology summaries that approach the quality of human-authored ones.
Code (0)
등록된 구현이 없습니다.
Tasks
Abstractive Text SummarizationFact CheckingReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Adversarial NLI for Factual Correctness in Text Summarisation Models
We apply the Adversarial NLI dataset to train the NLI model and show that the model has the potential to enhance factual correctness in abstract summarization. We follow the work of Falke et al. (2019), which rank multip…
FactReranker: Fact-guided Reranker for Faithful Radiology Report Summarization
Automatic radiology report summarization is a crucial clinical task, whose key challenge is to maintain factual accuracy between produced summaries and ground truth radiology findings. Existing research adopts reinforcem…
Graph GenerationFine-tuning BERT Models for Summarizing German Radiology Findings
Writing the conclusion section of radiology reports is essential for communicating the radiology findings and its assessment to physician in a condensed form. In this work, we employ a transformer-based Seq2Seq model for…
DecoderA Method to Automate the Discharge Summary Hospital Course for Neurology Patients
Generation of automated clinical notes have been posited as a strategy to mitigate physician burnout. In particular, an automated narrative summary of a patient's hospital stay could supplement the hospital course sectio…
DecoderGenerating medically-accurate summaries of patient-provider dialogue: A multi-stage approach using large language models
A medical provider's summary of a patient visit serves several critical purposes, including clinical decision-making, facilitating hand-offs between providers, and as a reference for the patient. An effective summary is …
Conversation SummarizationDecision MakingDialogue Understanding