Efficient Document Re-Ranking for Transformers by Precomputing Term Representations
Deep pretrained transformer networks are effective at various ranking tasks, such as question answering and ad-hoc document ranking. However, their computational expenses deem them cost-prohibitive in practice. Our proposed approach, called PreTTR (Precomputing Transformer Term Representations), considerably reduces the query-time latency of deep transformer networks (up to a 42x speedup on web document ranking) making these networks more practical to use in a real-time ranking scenario. Specifically, we precompute part of the document term representations at indexing time (without a query), and merge them with the query representation at query time to compute the final ranking score. Due to the large size of the token representations, we also propose an effective approach to reduce the storage requirement by training a compression layer to match attention scores. Our compression technique reduces the storage required up to 95% and it can be applied without a substantial degradation in ranking performance.
Code (1)
Tasks
Document RankingQuestion AnsweringRe-RankingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networks
Classical information retrieval (IR) methods, such as query likelihood and BM25, score documents independently w.r.t. each query term, and then accumulate the scores. Assuming query term independence allows precomputing …
Information RetrievalPassage RankingRe-RankingRetrievalTPRM: A Topic-based Personalized Ranking Model for Web Search
Ranking models have achieved promising results, but it remains challenging to design personalized ranking systems to leverage user profiles and semantic representations between queries and documents. In this paper, we pr…
Document RankingSDR: Efficient Neural Re-ranking using Succinct Document Representation
BERT based ranking models have achieved superior performance on various information retrieval tasks. However, the large number of parameters and complex self-attention operation come at a significant latency overhead. To…
Information RetrievalQuantizationRerankingRe-Ranking+1YNU-HPCC at SemEval-2019 Task 6: Identifying and Categorising Offensive Language on Twitter
This document describes the submission of team YNU-HPCC to SemEval-2019 for three Sub-tasks of Task 6: Sub-task A, Sub-task B, and Sub-task C. We have submitted four systems to identify and categorise offensive language.…
Deep LearningLearning to Match Using Local and Distributed Representations of Text for Web Search
Models such as latent semantic analysis and those based on neural embeddings learn distributed representations of text, and match the query against the document in the latent semantic space. In traditional information re…
Document RankingInformation RetrievalRetrieval