Check-COVID: Fact-Checking COVID-19 News Claims with Scientific Evidence
We present a new fact-checking benchmark, Check-COVID, that requires systems to verify claims about COVID-19 from news using evidence from scientific articles. This approach to fact-checking is particularly challenging as it requires checking internet text written in everyday language against evidence from journal articles written in formal academic language. Check-COVID contains 1, 504 expert-annotated news claims about the coronavirus paired with sentence-level evidence from scientific journal articles and veracity labels. It includes both extracted (journalist-written) and composed (annotator-written) claims. Experiments using both a fact-checking specific system and GPT-3.5, which respectively achieve F1 scores of 76.99 and 69.90 on this task, reveal the difficulty of automatically fact-checking both claim types and the importance of in-domain data for good performance. Our data and models are released publicly at https://github.com/posuer/Check-COVID.
Code (1)
Tasks
ArticlesFact CheckingSentenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Cross-lingual COVID-19 Fake News Detection
The COVID-19 pandemic poses a great threat to global public health. Meanwhile, there is massive misinformation associated with the pandemic which advocates unfounded or unscientific claims. Even major social media and ne…
Fact CheckingFake News DetectionMisinformationTwo Stage Transformer Model for COVID-19 Fake News Detection and Fact Checking
The rapid advancement of technology in online communication via social media platforms has led to a prolific rise in the spread of misinformation and fake news. Fake news is especially rampant in the current COVID-19 pan…
Fact CheckingFake News DetectionMisinformationNatural Language InferenceCoVERT: A Corpus of Fact-checked Biomedical COVID-19 Tweets
Over the course of the COVID-19 pandemic, large volumes of biomedical information concerning this new disease have been published on social media. Some of this information can pose a real danger to people's health, parti…
Fact CheckingMisinformationAraCOVID19-MFH: Arabic COVID-19 Multi-label Fake News and Hate Speech Detection Dataset
Along with the COVID-19 pandemic, an "infodemic" of false and misleading information has emerged and has complicated the COVID-19 response efforts. Social networking sites such as Facebook and Twitter have contributed la…
ArticlesDialect IdentificationFact CheckingFake News Detection+2Use of Retrieval-Augmented Large Language Model Agent for Long-Form COVID-19 Fact-Checking
The COVID-19 infodemic calls for scalable fact-checking solutions that handle long-form misinformation with accuracy and reliability. This study presents SAFE (system for accurate fact extraction and evaluation), an agen…