Target Conditioned Sampling: Optimizing Data Selection for Multilingual Neural Machine Translation
To improve low-resource Neural Machine Translation (NMT) with multilingual corpora, training on the most related high-resource language only is often more effective than using all data available (Neubig and Hu, 2018). However, it is possible that an intelligent data selection strategy can further improve low-resource NMT with data from other auxiliary languages. In this paper, we seek to construct a sampling distribution over all multilingual data, so that it minimizes the training loss of the low-resource language. Based on this formulation, we propose an efficient algorithm, Target Conditioned Sampling (TCS), which first samples a target sentence, and then conditionally samples its source sentence. Experiments show that TCS brings significant gains of up to 2 BLEU on three of four languages we test, with minimal training overhead.
Code (0)
등록된 구현이 없습니다.
Tasks
Low Resource Neural Machine TranslationLow-Resource Neural Machine TranslationLow Resource NMTMachine TranslationNMTSentenceTranslationSimilar Papers 제목 키워드 기반
Optimize Your Sampling: Tuned Diffusion Sampling with Bayesian Optimization
Sampling from a diffusion model typically requires many forward passes through a large neural network, making generation computationally expensive. While much work has focused on efficient solvers and samplers, comparati…
Text-to-Image GenerationOnline Learning-based Waveform Selection for Improved Vehicle Recognition in Automotive Radar
This paper describes important considerations and challenges associated with online reinforcement-learning based waveform selection for target identification in frequency modulated continuous wave (FMCW) automotive radar…
reinforcement-learningReinforcement Learning (RL)Thompson SamplingMeta-Aligner: Bidirectional Preference-Policy Optimization for Multi-Objective LLMs Alignment
Multi-Objective Alignment aims to align Large Language Models (LLMs) with diverse and often conflicting human values by optimizing multiple objectives simultaneously. Existing methods predominantly rely on static prefere…
Response GenerationEvidence-Driven Dynamic Visual Selector for Efficient Long Video Understanding
Recent advancements in MLLM-based long-form video understanding have mitigated inference-time computational cost and limited context lengths by selecting query-relevant frames. However, existing approaches predominantly …
Global Resolution: Optimal Multi-Draft Speculative Sampling via Convex Minimization
Speculative sampling reduces the latency of autoregressive decoding for target model LLMs without sacrificing inference quality, by using a cheap draft model to suggest a candidate token and a verification criterion to a…