Modeling Document Interactions for Learning to Rank with Regularized Self-Attention
Learning to rank is an important task that has been successfully deployed in many real-world information retrieval systems. Most existing methods compute relevance judgments of documents independently, without holistically considering the entire set of competing documents. In this paper, we explore modeling documents interactions with self-attention based neural networks. Although self-attention networks have achieved state-of-the-art results in many NLP tasks, we find empirically that self-attention provides little benefit over baseline neural learning to rank architecture. To improve the learning of self-attention weights, We propose simple yet effective regularization terms designed to model interactions between documents. Evaluations on publicly available Learning to Rank (LETOR) datasets show that training self-attention network with our proposed regularization terms can significantly outperform existing learning to rank methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Information RetrievalLearning-To-RankRetrievalSimilar Papers 제목 키워드 기반
Self-Attentive Document Interaction Networks for Permutation Equivariant Ranking
How to leverage cross-document interactions to improve ranking performance is an important topic in information retrieval (IR) research. However, this topic has not been well-studied in the learning-to-rank setting and m…
Information RetrievalLearning-To-RankRetrievalSetRank: Learning a Permutation-Invariant Ranking Model for Information Retrieval
In learning-to-rank for information retrieval, a ranking model is automatically learned from the data and then utilized to rank the sets of retrieved documents. Therefore, an ideal ranking model would be a mapping from a…
Information RetrievalLearning-To-RankRetrievalA Multi-Agent Perspective on Modern Information Retrieval
The rise of large language models (LLMs) has introduced a new era in information retrieval (IR), where queries and documents that were once assumed to be generated exclusively by humans can now also be created by automat…
Information RetrievalRetrievalCo-BERT: A Context-Aware BERT Retrieval Model Incorporating Local and Query-specific Context
BERT-based text ranking models have dramatically advanced the state-of-the-art in ad-hoc retrieval, wherein most models tend to consider individual query-document pairs independently. In the mean time, the importance and…
Learning-To-RankRe-RankingRetrievalSupervised Quantile Normalization for Low Rank Matrix Factorization
Low rank matrix factorization is a fundamental building block in machine learning, used for instance to summarize gene expression profile data or word-document counts. To be robust to outliers and differences in scale ac…