Calibration of Encoder Decoder Models for Neural Machine Translation
We study the calibration of several state of the art neural machine translation(NMT) systems built on attention-based encoder-decoder models. For structured outputs like in NMT, calibration is important not just for reliable confidence with predictions, but also for proper functioning of beam-search inference. We show that most modern NMT models are surprisingly miscalibrated even when conditioned on the true previous tokens. Our investigation leads to two main reasons -- severe miscalibration of EOS (end of sequence marker) and suppression of attention uncertainty. We design recalibration methods based on these signals and demonstrate improved accuracy, better sequence-level calibration, and more intuitive results from beam-search.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderMachine TranslationNMTTranslationSimilar Papers 제목 키워드 기반
Implicit Distortion and Fertility Models for Attention-based Encoder-Decoder NMT Model
Neural machine translation has shown very promising results lately. Most NMT models follow the encoder-decoder framework. To make encoder-decoder models more flexible, attention mechanism was introduced to machine transl…
AttributeDecoderImage CaptioningMachine Translation+4Machine Translation with Large Language Models: Decoder Only vs. Encoder-Decoder
This project, titled "Machine Translation with Large Language Models: Decoder-only vs. Encoder-Decoder," aims to develop a multilingual machine translation (MT) model. Focused on Indian regional languages, especially Tel…
DecoderMachine TranslationTranslationOn the Properties of Neural Machine Translation: Encoder-Decoder Approaches
Neural machine translation is a relatively new approach to statistical machine translation based purely on neural networks. The neural machine translation models often consist of an encoder and a decoder. The encoder ext…
DecoderMachine TranslationSentenceTranslationIs Encoder-Decoder Redundant for Neural Machine Translation?
Encoder-decoder architecture is widely adopted for sequence-to-sequence modeling tasks. For machine translation, despite the evolution from long short-term memory networks to Transformer networks, plus the introduction a…
DecoderLanguage ModelingLanguage ModellingMachine Translation+2Multilingual Neural Machine Translation With the Right Amount of Sharing
Large multilingual Transformer-based machine translation models have had a pivotal role in making translation systems available for hundreds of languages with good zero-shot translation performance. One such example is t…
DecoderMachine TranslationNMTTranslation