Learning Decoupled Retrieval Representation for Nearest Neighbour Neural Machine Translation
K-Nearest Neighbor Neural Machine Translation (kNN-MT) successfully incorporates external corpus by retrieving word-level representations at test time. Generally, kNN-MT borrows the off-the-shelf context representation in the translation task, e.g., the output of the last decoder layer, as the query vector of the retrieval task. In this work, we highlight that coupling the representations of these two tasks is sub-optimal for fine-grained retrieval. To alleviate it, we leverage supervised contrastive learning to learn the distinctive retrieval representation derived from the original context representation. We also propose a fast and effective approach to constructing hard negative samples. Experimental results on five domains show that our approach improves the retrieval accuracy and BLEU score compared to vanilla kNN-MT.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningDecoderMachine TranslationRetrievalTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
k-Nearest Neighbour Classifiers: 2nd Edition (with Python examples)
Perhaps the most straightforward classifier in the arsenal or machine learning techniques is the Nearest Neighbour Classifier -- classification is achieved by identifying the nearest neighbours to a query example and usi…
ClassificationGeneral ClassificationRetrievalTime Series+1MicroNN: An On-device Disk-resident Updatable Vector Database
Nearest neighbour search over dense vector collections has important applications in information retrieval, retrieval augmented generation (RAG), and content ranking. Performing efficient search over large vector collect…
AttributeInformation RetrievalRAGRetrieval+1Robust Nearest Neighbour Retrieval Using Targeted Manifold Manipulation
Nearest-neighbour retrieval is central to classification and explainable-AI pipelines, but current practice relies on hand-tuning feature layers and distance metrics. We propose Targeted Manifold Manipulation-Nearest Nei…
Distributional Approximate Nearest Neighbour Search for Uncertainty-Aware Retrieval
Approximate Nearest Neighbour search indices form the backbone of real-world recommender systems, enabling real-time candidate retrieval over million-item catalogues. Typically, a single point estimate embedding is learn…
RelocNet: Continuous Metric Learning Relocalisation using Neural Nets
We propose a method of learning suitable convolutional representations for camera pose retrieval based on nearest neighbour matching and continuous metric learning-based feature descriptors. We introduce informatio…
Metric LearningPose RetrievalRetrieval