The Automatic Verification of Image-Text Claims (AVerImaTeC) Shared Task
The Automatic Verification of Image-Text Claims (AVerImaTeC) shared task aims to advance system development for retrieving evidence and verifying real-world image-text claims. Participants were allowed to either employ external knowledge sources, such as web search engines, or leverage the curated knowledge store provided by the organizers. System performance was evaluated using the AVerImaTeC score, defined as a conditional verdict accuracy in which a verdict is considered correct only when the associated evidence score exceeds a predefined threshold. The shared task attracted 14 submissions during the development phase and 6 submissions during the testing phase. All participating systems in the testing phase outperformed the baseline provided. The winning team, HUMANE, achieved an AVerImaTeC score of 0.5455. This paper provides a detailed description of the shared task, presents the complete evaluation results, and discusses key insights and lessons learned.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
VILLAIN at AVerImaTeC: Verifying Image-Text Claims via Multi-Agent Collaboration
This paper describes VILLAIN, a multimodal fact-checking system that verifies image-text claims through prompt-based multi-agent collaboration. For the AVerImaTeC shared task, VILLAIN employs vision-language model agents…
AIC CTU@AVerImaTeC: dual-retriever RAG for image-text fact checking
In this paper, we present our 3rd place system in the AVerImaTeC shared task, which combines our last year's retrieval-augmented generation (RAG) pipeline with a reverse image search (RIS) module. Despite its simplicity,…
Image RetrievalFact CheckingNeural Machine Translation for Fact-checking Temporal Claims
Computational fact-checking aims at supporting the verification process of textual claims by exploiting trustworthy sources. However, there are large classes of complex claims that cannot be automatically verified, for i…
Fact CheckingMachine TranslationTime SeriesTime Series Analysis+1Automatic Verification of Data Summaries
We present a generic method to compute thefactual accuracy of a generated data summarywith minimal user effort. We look at the prob-lem as a fact-checking task to verify the nu-merical claims in the text. The verificatio…
Fact CheckingStance Prediction and Claim Verification: An Arabic Perspective
This work explores the application of textual entailment in news claim verification and stance prediction using a new corpus in Arabic. The publicly available corpus comes in two perspectives: a version consisting of 4,5…
Claim VerificationNatural Language InferencePredictionWorld Knowledge