Learning to Match Using Local and Distributed Representations of Text for Web Search
Models such as latent semantic analysis and those based on neural embeddings learn distributed representations of text, and match the query against the document in the latent semantic space. In traditional information retrieval models, on the other hand, terms have discrete or local representations, and the relevance of a document is determined by the exact matches of query terms in the body text. We hypothesize that matching with distributed representations complements matching with traditional local representations, and that a combination of the two is favorable. We propose a novel document ranking model composed of two separate deep neural networks, one that matches the query and the document using a local representation, and another that matches the query and the document using learned distributed representations. The two networks are jointly trained as part of a single neural network. We show that this combination or `duet' performs significantly better than either neural network individually on a Web page ranking task, and also significantly outperforms traditional baselines and other recently proposed models based on neural networks.
Code (1)
Tasks
Document RankingInformation RetrievalRetrievalSimilar Papers 제목 키워드 기반
Evaluating Distributed Representations for Multi-Level Lexical Semantics: A Research Proposal
Modern neural networks (NNs), trained on extensive raw sentence data, construct distributed representations by compressing individual words into dense, continuous, high-dimensional vectors. These representations are expe…
SentenceFederated Contrastive Representation Learning with Feature Fusion and Neighborhood Matching
Federated learning (FL) enables distributed clients to learn a shared model for prediction while keeping the training data local on each client. However, existing FL requires fully-labeled data for training, which is inc…
Contrastive LearningFederated LearningRepresentation LearningSelf-Supervised LearningDecentralized Unsupervised Learning of Visual Representations
Collaborative learning enables distributed clients to learn a shared model for prediction while keeping the training data local on each client. However, existing collaborative learning methods require fully-labeled data …
Contrastive LearningFederated LearningRepresentation LearningSelf-Supervised LearningLearning Semantic Similarity for Very Short Texts
Levering data on social media, such as Twitter and Facebook, requires information retrieval algorithms to become able to relate very short text fragments to each other. Traditional text similarity methods such as tf-idf …
Information RetrievalRetrievalSemantic SimilaritySemantic Textual Similarity+3Local vs distributed representations: What is the right basis for interpretability?
Much of the research on the interpretability of deep neural networks has focused on studying the visual features that maximally activate individual neurons. However, recent work has cast doubts on the usefulness of such …