Subword-Delimited Downsampling for Better Character-Level Translation
Subword-level models have been the dominant paradigm in NLP. However, character-level models have the benefit of seeing each character individually, providing the model with more detailed information that ultimately could lead to better models. Recent works have shown character-level models to be competitive with subword models, but costly in terms of time and computation. Character-level models with a downsampling component alleviate this, but at the cost of quality, particularly for machine translation. This work analyzes the problems of previous downsampling methods and introduces a novel downsampling method which is informed by subwords. This new downsampling method not only outperforms existing downsampling methods, showing that downsampling characters can be done without sacrificing quality, but also leads to promising performance compared to subword models for translation.
Code (1)
Tasks
Machine TranslationTranslationSimilar Papers 제목 키워드 기반
Patching Leaks in the Charformer for Efficient Character-Level Generation
Character-based representations have important advantages over subword-based ones for morphologically rich languages. They come with increased robustness to noisy input and do not need a separate tokenization step. Howev…
DecoderNMTTranslationPatching Leaks in the Charformer for Generative Tasks
Character-based representations have important advantages over subword-based ones, including increased robustness to noisy input and removing the need of tokenization preprocessing. However, they also have a crucial disa…
DecoderNMTWhy don't people use character-level machine translation?
We present a literature and empirical survey that critically assesses the state of the art in character-level modeling for machine translation (MT). Despite evidence in the literature that character-level systems are com…
Machine TranslationTranslationWhy don’t people use character-level machine translation?
We present a literature and empirical survey that critically assesses the state of the art in character-level modeling for machine translation (MT). Despite evidence in the literature that character-level systems are com…
Machine TranslationTranslationWhen is Char Better Than Subword: A Systematic Study of Segmentation Algorithms for Neural Machine Translation
Subword segmentation algorithms have been a \textit{de facto} choice when building neural machine translation systems. However, most of them need to learn a segmentation model based on some heuristics, which may produce …
Machine TranslationNMTSegmentationTranslation