FaBULOUS: Fact-checking Based on Understanding of Language Over Unstructured and Structured information
As part of the FEVEROUS shared task, we developed a robust and finely tuned architecture to handle the joint retrieval and entailment on text data as well as structured data like tables. We proposed two training schemes to tackle the hurdles inherent to multi-hop multi-modal datasets. The first one allows having a robust retrieval of full evidence sets, while the second one enables entailment to take full advantage of noisy evidence inputs. In addition, our work has revealed important insights and potential avenue of research for future improvement on this kind of dataset. In preliminary evaluation on the FEVEROUS shared task test set, our system achieves 0.271 FEVEROUS score, with 0.4258 evidence recall and 0.5607 entailment accuracy.
Code (0)
등록된 구현이 없습니다.
Tasks
Fact CheckingRetrievalSimilar Papers 제목 키워드 기반
Generative Large Language Models in Automated Fact-Checking: A Survey
The dissemination of false information on online platforms presents a serious societal challenge. While manual fact-checking remains crucial, Large Language Models (LLMs) offer promising opportunities to support fact-che…
Fact CheckingSurveyEvidence-based Interpretable Open-domain Fact-checking with Large Language Models
Universal fact-checking systems for real-world claims face significant challenges in gathering valid and sufficient real-time evidence and making reasoned decisions. In this work, we introduce the Open-domain Explainable…
Fact CheckingvalidThe Perils & Promises of Fact-checking with Large Language Models
Automated fact-checking, using machine learning to verify claims, has grown vital as misinformation spreads beyond human fact-checking capacity. Large Language Models (LLMs) like GPT-4 are increasingly trusted to write a…
ArticlesFact CheckingMisinformationAutomated Fact Checking: Task formulations, methods and future directions
The recently increased focus on misinformation has stimulated research in fact checking, the task of assessing the truthfulness of a claim. Research in automating this task has been conducted in a variety of disciplines …
ArticlesFact CheckingMisinformationLogical Consistency of Large Language Models in Fact-checking
In recent years, large language models (LLMs) have demonstrated significant success in performing varied natural language tasks such as language translation, question-answering, summarizing, fact-checking, etc. Despite L…
Fact CheckingHallucinationKnowledge GraphsLogical Reasoning+3