Building Retrieval Systems for the ClueWeb22-B Corpus

2024-02-06 · Harshit Mehrotra, Jamie Callan, Zhen Fan

The ClueWeb22 dataset containing nearly 10 billion documents was released in 2022 to support academic and industry research. The goal of this project was to build retrieval baselines for the English section of the "super head" part (category B) of this dataset. These baselines can then be used by the research community to compare their systems and also to generate data to train/evaluate new retrieval and ranking algorithms. The report covers sparse and dense first stage retrievals as well as neural rerankers that were implemented for this dataset. These systems are available as a service on a Carnegie Mellon University cluster.

📄 PDF Abstract BibTeX arXiv:2402.04357

Code (0)

등록된 구현이 없습니다.

Tasks

Retrieval

Methods 이 논문이 사용한 방법론

Golden Queue Managers 설명 없음

ClueWeb22: 10 Billion Web Documents with Visual and Semantic Information

2022-11-29 · Arnold Overwijk, Chenyan Xiong, Xiao Liu, Cameron VandenBerg 외

ClueWeb22, the newest iteration of the ClueWeb line of datasets, provides 10 billion web pages affiliated with rich information. Its design was influenced by the need for a high quality, large scale web corpus to support…

document understanding Retrieval

Building a 70 billion word corpus of English from ClueWeb

2012-05-01 · LREC 2012 5 · Jan Pomik{\'a}lek, Milo{\v{s}} Jakub{\'\i}{\v{c}}ek, Pavel Rychl{\'y}

This work describes the process of creation of a 70 billion word text corpus of English. We used an existing language resource, namely the ClueWeb09 dataset, as source for the corpus data. Processing such a vast amount o…

Machine Translation Management Part-Of-Speech Tagging Speech Recognition

MARVEL: Unlocking the Multi-Modal Capability of Dense Retrieval via Visual Module Plugin

2023-10-21 · Tianshuo Zhou, Sen Mei, Xinze Li, Zhenghao Liu 외

This paper proposes Multi-modAl Retrieval model via Visual modulE pLugin (MARVEL), which learns an embedding space for queries and multi-modal documents to conduct retrieval. MARVEL encodes queries and multi-modal docume…

Language Modelling Retrieval Text Retrieval

Quasar: Datasets for Question Answering by Search and Reading

2017-07-12 · Bhuwan Dhingra, Kathryn Mazaitis, William W. Cohen

We present two new large-scale datasets aimed at evaluating systems designed to comprehend a natural language query and extract its answer from a large corpus of text. The Quasar-S dataset consists of 37000 cloze-style (…

Question Answering Retrieval

WIKIR: A Python toolkit for building a large-scale Wikipedia-based English Information Retrieval Dataset

2019-12-04 · LREC 2020 5 · Jibril Frej, Didier Schwab, Jean-Pierre Chevallet

Over the past years, deep learning methods allowed for new state-of-the-art results in ad-hoc information retrieval. However such methods usually require large amounts of annotated data to be effective. Since most standa…

Ad-Hoc Information Retrieval Information Retrieval Retrieval