Knowledge Distillation Improves Stability in Retranslation-based Simultaneous Translation
In simultaneous translation, the \emph{retranslation} approach has the advantage of requiring no modifications to the inference engine. However in order to reduce the undesirable instability (flicker) in the output, previous work has resorted to increasing the latency through masking, and introducing specialised inference, losing the simplicity of the approach. In this paper, we argue that the flicker is caused by both non-monotonicity of the training data, and by non-determinism of the resulting model. Both of these can be addressed using knowledge distillation. We evaluate our approach using simultaneously interpreted test sets for English-German and English-Czech and demonstrate that the distilled models have an improved flicker-latency tradeoff, with quality similar to the original.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge DistillationTranslationSimilar Papers 제목 키워드 기반
Does partial pretranslation can improve low ressourced-languages pairs?
We study the effects of a local and punctual pretranslation of the source corpus on the performance of a Transformer translation model. The pretranslations are performed at the morphological (morpheme translation), lexic…
TranslationWord TranslationSynergy Between the Strong and the Weak: Spiking Neural Networks are Inherently Self-Distillers
Brain-inspired spiking neural networks (SNNs) promise to be a low-power alternative to computationally intensive artificial neural networks (ANNs), although performance gaps persist. Recent studies have improved the perf…
Knowledge DistillationAdversarial RobustnessSimultaneous Translation for Unsegmented Input: A Sliding Window Approach
In the cascaded approach to spoken language translation (SLT), the ASR output is typically punctuated and segmented into sentences before being passed to MT, since the latter is typically trained on written text. However…
SentenceTranslation