Corpora for Cross-Language Information Retrieval in Six Less-Resourced Languages
The Machine Translation for English Retrieval of Information in Any Language (MATERIAL) research program, sponsored by the Intelligence Advanced Research Projects Activity (IARPA), focuses on rapid development of end-to-end systems capable of retrieving foreign language speech and text documents relevant to different types of English queries that may be further restricted by domain. Those systems also provide evidence of relevance of the retrieved content in the form of English summaries. The program focuses on Less-Resourced Languages and provides its performer teams very limited amounts of annotated training data. This paper describes the corpora that were created for system development and evaluation for the six languages released by the program to date: Tagalog, Swahili, Somali, Lithuanian, Bulgarian and Pashto. The corpora include build packs to train Machine Translation and Automatic Speech Recognition systems; document sets in three text and three speech genres annotated for domain and partitioned for analysis, development and evaluation; and queries of several types together with corresponding binary relevance judgments against the entire set of documents. The paper also describes a detection metric called Actual Query Weighted Value developed by the program to evaluate end-to-end system performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Information RetrievalMachine TranslationRetrievalspeech-recognitionSpeech RecognitionTranslationSimilar Papers 제목 키워드 기반
Language Agnostic Multilingual Information Retrieval with Contrastive Learning
Multilingual information retrieval (IR) is challenging since annotated training data is costly to obtain in many languages. We present an effective method to train multilingual IR systems when only English IR training da…
Contrastive LearningCross-Lingual TransferInformation RetrievalPretrained Multilingual Language Models+3SHRAG: AFrameworkfor Combining Human-Inspired Search with RAG
Retrieval-Augmented Generation (RAG) is gaining recognition as one of the key technological axes for next generation information retrieval, owing to its ability to mitigate the hallucination phenomenon in Large Language …
Cross-Lingual Question AnsweringNatural Language QueriesInformation RetrievalWeakly Supervised Attentional Model for Low Resource Ad-hoc Cross-lingual Information Retrieval
We propose a weakly supervised neural model for Ad-hoc Cross-lingual Information Retrieval (CLIR) from low-resource languages. Low resource languages often lack relevance annotations for CLIR, and when available the trai…
Cross-Lingual Information RetrievalInformation RetrievalMachine TranslationRetrieval+2CoIR: A Comprehensive Benchmark for Code Information Retrieval Models
Despite the substantial success of Information Retrieval (IR) in various NLP tasks, most IR systems predominantly handle queries and corpora in natural language, neglecting the domain of code retrieval. Code retrieval is…
BenchmarkingCode SearchInformation RetrievalRetrievalEnhancing Retrieval-Augmented LMs with a Two-stage Consistency Learning Compressor
Despite the prevalence of retrieval-augmented language models (RALMs), the seamless integration of these models with retrieval mechanisms to enhance performance in document-based tasks remains challenging. While some pos…
Question AnsweringRAGRetrievalRetrieval-augmented Generation