An Evaluation of Information Extraction Tools for Identifying Health Claims in News Headlines
This study evaluates the performance of four information extraction tools (extractors) on identifying health claims in health news headlines. A health claim is defined as a triplet: IV (what is being manipulated), DV (what is being measured) and their relation. Tools that can identify health claims provide the foundation for evaluating the accuracy of these claims against authoritative resources. The evaluation result shows that 26{\%} headlines do not in-clude health claims, and all extractors face difficulty separating them from the rest. For those with health claims, OPENIE-5.0 performed the best with F-measure at 0.6 level for ex-tracting {``}IV-relation-DV{''}. However, the characteristic linguistic structures in health news headlines, such as incomplete sentences and non-verb relations, pose particular challenge to existing tools.
Code (0)
등록된 구현이 없습니다.
Tasks
RelationRelation ExtractionTripletSimilar Papers 제목 키워드 기반
Identifying Health Risks from Family History: A Survey of Natural Language Processing Techniques
Electronic health records include information on patients' status and medical history, which could cover the history of diseases and disorders that could be hereditary. One important use of family history information is …
I-SIRch: AI-Powered Concept Annotation Tool For Equitable Extraction And Analysis Of Safety Insights From Maternity Investigations
Maternity care is a complex system involving treatments and interactions between patients, providers, and the care environment. To improve patient safety and outcomes, understanding the human factors (e.g. individuals de…
Evaluating Large Language Models for Public Health Classification and Extraction Tasks
Advances in Large Language Models (LLMs) have led to significant interest in their potential to support human experts across a range of domains, including public health. In this work we present automated evaluations of L…
In-Context LearningHunFlair2 in a cross-corpus evaluation of biomedical named entity recognition and normalization tools
With the exponential growth of the life science literature, biomedical text mining (BTM) has become an essential technology for accelerating the extraction of insights from publications. Identifying named entities (e.g.,…
Cross-corpusnamed-entity-recognitionNamed Entity RecognitionEntity Graph Extraction from Legal Acts -- a Prototype for a Use Case in Policy Design Analysis
This paper presents research on a prototype developed to serve the quantitative study of public policy design. This sub-discipline of political science focuses on identifying actors, relations between them, and tools at …