Ad-hoc Document Retrieval using Weak-Supervision with BERT and GPT2
We describe a weakly-supervised method for training deep learning models for the task of ad-hoc document retrieval. Our method is based on generative and discriminative models that are trained using weak-supervision just from the documents in the corpus. We present an end-to-end retrieval system that starts with traditional information retrieval methods, followed by two deep learning re-rankers. We evaluate our method on three different datasets: a COVID-19 related scientific literature dataset and two news datasets. We show that our method outperforms state-of-the-art methods; this without the need for the expensive process of manually labeling data.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningInformation RetrievalRetrievalSimilar Papers 제목 키워드 기반
Cross-lingual Information Retrieval with BERT
Multiple neural language models have been developed recently, e.g., BERT and XLNet, and achieved impressive results in various NLP tasks including sentence classification, question answering and document ranking. In this…
Cross-Lingual Information RetrievalDocument RankingInformation RetrievalLanguage Modeling+5An In-depth Analysis of Passage-Level Label Transfer for Contextual Document Ranking
Pre-trained contextual language models such as BERT, GPT, and XLnet work quite well for document retrieval tasks. Such models are fine-tuned based on the query-document/query-passage level relevance labels to capture the…
Document RankingRetrievalWeakly Supervised Label Smoothing
We study Label Smoothing (LS), a widely used regularization technique, in the context of neural learning to rank (L2R) models. LS combines the ground-truth labels with a uniform distribution, encouraging the model to be …
Learning-To-RankPassage RetrievalRetrievalSelective Weak Supervision for Neural Information Retrieval
This paper democratizes neural information retrieval to scenarios where large scale relevance training signals are not available. We revisit the classic IR intuition that anchor-document relations approximate query-docum…
Information RetrievalLearning-To-RankRetrievalExtracting Appointment Spans from Medical Conversations
Extracting structured information from medical conversations can reduce the documentation burden for doctors and help patients follow through with their care plan. In this paper, we introduce a novel task of extracting a…