Multilingual pre-training with Language and Task Adaptation for Multilingual Text Style Transfer
We exploit the pre-trained seq2seq model mBART for multilingual text style transfer. Using machine translated data as well as gold aligned English sentences yields state-of-the-art results in the three target languages we consider. Besides, in view of the general scarcity of parallel data, we propose a modular approach for multilingual formality transfer, which consists of two training strategies that target adaptation to both language and task. Our approach achieves competitive performance without monolingual task-specific parallel data and can be applied to other style transfer tasks as well as to other languages.
Code (0)
등록된 구현이 없습니다.
Tasks
Style TransferText Style TransferMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Multilingual Pre-training with Language and Task Adaptation for Multilingual Text Style Transfer
We exploit the pre-trained seq2seq model mBART for multilingual text style transfer. Using machine translated data as well as gold aligned English sentences yields state-of-the-art results in the three target languages w…
Style TransferText Style TransferAdaptation of Deep Bidirectional Multilingual Transformers for Russian Language
The paper introduces methods of adaptation of multilingual masked language models for a specific language. Pre-trained bidirectional language models show state-of-the-art performance on a wide range of tasks including re…
Natural Language InferenceParaphrase IdentificationQuestion AnsweringReading Comprehension+2Leveraging Routing Dynamics in Mixture-of-Experts Models for Efficient Language Adaptation
Mixture-of-Experts (MoE) models are widely used to scale language models, yet their expert routing behavior and adaptation in a multilingual setting remain underexplored. In this work, we study multilingual routing dynam…
Targeted Multilingual Adaptation for Low-resource Language Families
The "massively-multilingual" training of multilingual models is known to limit their utility in any one language, and they perform particularly poorly on low-resource languages. However, there is evidence that low-resour…
XLM-RAdapting Multilingual Neural Machine Translation to Unseen Languages
Multilingual Neural Machine Translation (MNMT) for low-resource languages (LRL) can be enhanced by the presence of related high-resource languages (HRL), but the relatedness of HRL usually relies on predefined linguistic…
Data AugmentationMachine TranslationNMTTranslation