Scalable Fact-checking with Human-in-the-Loop
Researchers have been investigating automated solutions for fact-checking in a variety of fronts. However, current approaches often overlook the fact that the amount of information released every day is escalating, and a large amount of them overlap. Intending to accelerate fact-checking, we bridge this gap by grouping similar messages and summarizing them into aggregated claims. Specifically, we first clean a set of social media posts (e.g., tweets) and build a graph of all posts based on their semantics; Then, we perform two clustering methods to group the messages for further claim summarization. We evaluate the summaries both quantitatively with ROUGE scores and qualitatively with human evaluation. We also generate a graph of summaries to verify that there is no significant overlap among them. The results reduced 28,818 original messages to 700 summary claims, showing the potential to speed up the fact-checking process by organizing and selecting representative claims from massive disorganized and redundant messages.
Code (1)
Tasks
Fact CheckingSimilar Papers 제목 키워드 기반
The State of Human-centered NLP Technology for Fact-checking
Misinformation threatens modern society by promoting distrust in science, changing narratives in public health, heightening social polarization, and disrupting democratic elections and financial markets, among a myriad o…
Explainable ModelsFact CheckingMisinformationHuman-in-the-loop Evaluation for Early Misinformation Detection: A Case Study of COVID-19 Treatments
We present a human-in-the-loop evaluation framework for fact-checking novel misinformation claims and identifying social media messages that support them. Our approach extracts check-worthy claims, which are aggregated a…
Fact CheckingMisinformationReinforcement Learning-based Knowledge Graph Reasoning for Explainable Fact-checking
Fact-checking is a crucial task as it ensures the prevention of misinformation. However, manual fact-checking cannot keep up with the rate at which false information is generated and disseminated online. Automated fact-c…
Fact CheckingMisinformationreinforcement-learningReinforcement Learning+1Assessing the Potential of Generative Agents in Crowdsourced Fact-Checking
The growing spread of online misinformation has created an urgent need for scalable, reliable fact-checking solutions. Crowdsourced fact-checking - where non-experts evaluate claim veracity - offers a cost-effective alte…
Decision MakingFact CheckingInformativenessMisinformationThe Role of Context in Detecting Previously Fact-Checked Claims
Recent years have seen the proliferation of disinformation and misinformation online, thanks to the freedom of expression on the Internet and to the rise of social media. Two solutions were proposed to address the proble…
Fact CheckingMisinformation