Unsupervised Fact Checking by Counter-Weighted Positive and Negative Evidential Paths in A Knowledge Graph
Misinformation spreads across media, community, and knowledge graphs in the Web by not only human agents but also information extraction algorithms that extract factual statements from unstructured textual data to populate the existing knowledge graphs. Traditional fact checking by experts or crowds is increasingly difficult to keep pace with the volume of newly created misinformation in the Web. Therefore, it is important and necessary to enhance the computational ability to determine whether a given factual statement is truthful or not. We view this problem as a truth scoring task in a knowledge graph. We present a novel rule-based approach that finds positive and negative evidential paths in a knowledge graph for a given factual statement and calculates a truth score for the given statement by unsupervised ensemble of the found positive and negative evidential paths. For example, we can determine the factual statement {`}United States is the birth place of Barack Obama{''} as truthful if there is the positive evidential path (Barack Obama, birthPlace, Hawaii) ∧ (Hawaii, country, United States) in a knowledge graph. For another example, we can determine the factual statement {`}Canada is the nationality of Barack Obama{''} as untruthful if there is the negative evidential path (Barack Obama, nationality, United States) ∧ (United States, {\mbox{$\neq$}}, Canada) in a knowledge graph. For evaluating on a real-world situation, we constructed an evaluation dataset by labeling truth or untruth label on factual statements that were extracted from Wikipedia texts by using the state-of-the-art BERT-based information extraction system. Our evaluation results show that our approach outperforms the state-of-the-art unsupervised approaches significantly by up to 0.12 AUC-ROC and even outperforms the supervised approach by up to 0.05 AUC-ROC not only in our dataset but also in the two different standard datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Fact CheckingKnowledge GraphsMisinformationSimilar Papers 제목 키워드 기반
Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformation
Misinformation emerges in times of uncertainty when credible information is limited. This is challenging for NLP-based fact-checking as it relies on counter-evidence, which may not yet be available. Despite increasing in…
ArticlesFact CheckingMisinformation+VeriRel: Verification Feedback to Enhance Document Retrieval for Scientific Fact Checking
Identification of appropriate supporting evidence is critical to the success of scientific fact checking. However, existing approaches rely on off-the-shelf Information Retrieval algorithms that rank documents based on r…
Information RetrievalDocument RankingFact CheckingAsk to Know More: Generating Counterfactual Explanations for Fake Claims
Automated fact checking systems have been proposed that quickly provide veracity prediction at scale to mitigate the negative influence of fake news on people and on public opinion. However, most studies focus on veracit…
ArticlescounterfactualCounterfactual ExplanationFact Checking+1Generating Fluent Fact Checking Explanations with Unsupervised Post-Editing
Fact-checking systems have become important tools to verify fake and misguiding news. These systems become more trustworthy when human-readable explanations accompany the veracity labels. However, manual collection of su…
Explanation GenerationExtractive SummarizationFact CheckingSentenceAC2L-GAD: Active Counterfactual Contrastive Learning for Graph Anomaly Detection
Graph anomaly detection aims to identify abnormal patterns in networks, but faces significant challenges from label scarcity and extreme class imbalance. While graph contrastive learning offers a promising unsupervised s…
Graph Anomaly DetectionContrastive Learning