Training and Inference Methods for High-Coverage Neural Machine Translation

2020-07-01 · WS 2020 7 · Michael Yang, Yixin Liu, Rahul Mayuranath

In this paper, we introduce a system built for the Duolingo Simultaneous Translation And Paraphrase for Language Education (STAPLE) shared task at the 4th Workshop on Neural Generation and Translation (WNGT 2020). We participated in the English-to-Japanese track with a Transformer model pretrained on the JParaCrawl corpus and fine-tuned in two steps on the JESC corpus and then the (smaller) Duolingo training corpus. First, during training, we find it is essential to deliberately expose the model to higher-quality translations more often during training for optimal translation performance. For inference, encouraging a small amount of diversity with Diverse Beam Search to improve translation coverage yielded marginal improvement over regular Beam Search. Finally, using an auxiliary filtering model to filter out unlikely candidates from Beam Search improves performance further. We achieve a weighted F1 score of 27.56{\%} on our own test set, outperforming the STAPLE AWS translations baseline score of 4.31{\%}.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

DiversityMachine Translation Translation Vocal Bursts Intensity Prediction

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Position-Wise Feed-Forward Layer 설명 없음

Residual Connection 설명 없음

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Multi-Head Attention 설명 없음

Adam 설명 없음

ReLU How Do I Communicate to Expedia? How Do I Communicate to Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Live Support & Special Travel…

Likelihood-Free Frequentist Inference: Bridging Classical Statistics and Machine Learning for Reliable Simulator-Based Inference

2021-07-08 · Niccolò Dalmasso, Luca Masserano, David Zhao, Rafael Izbicki 외

Many areas of science rely on simulators that implicitly encode intractable likelihood functions of complex systems. Classical statistical methods are poorly suited for these so-called likelihood-free inference (LFI) set…

Diagnostic Open-Ended Question Answering valid

Error-quantified Conformal Inference for Time Series

2025-02-02 · Junxi Wu, Dongjian Hu, Yajie Bao, Shu-Tao Xia 외

Uncertainty quantification in time series prediction is challenging due to the temporal dependence and distribution shift on sequential data. Conformal inference provides a pivotal and flexible instrument for assessing t…

Prediction Time Series Time Series Prediction Uncertainty Quantification+1

Machine-Assisted Map Editing

2019-06-17 · Favyen Bastani, Songtao He, Sofiane Abbar, Mohammad Alizadeh 외

Mapping road networks today is labor-intensive. As a result, road maps have poor coverage outside urban centers in many countries. Systems to automatically infer road network graphs from aerial imagery and GPS trajectori…

graph construction

Conformal Prediction for Nonparametric Instrumental Regression

2026-03-26 · Masahiro Kato arxiv

We propose a method for constructing distribution-free prediction intervals in nonparametric instrumental variable regression (NPIV), with finite-sample coverage guarantees. Building on the conditional guarantee framewor…

StackTok: Accelerating VLMs Inference with Budget-Adaptive Visual Token Selection

2026-09-15 · Zhenbin Wang, Lei Zhang, Lituan Wang, Wei Huang 외 arxiv

Increasing image resolution produces ever-longer visual-token sequences in vision-language models (VLMs), substantially raising their inference cost. To reduce this overhead without retraining, existing methods select co…