Multi-level Alignment Pretraining for Multi-lingual Semantic Parsing
In this paper, we present a multi-level alignment pretraining method in a unified architecture formulti-lingual semantic parsing. In this architecture, we use an adversarial training method toalign the space of different languages and use sentence level and word level parallel corpus assupervision information to align the semantic of different languages. Finally, we jointly train themulti-level alignment and semantic parsing tasks. We conduct experiments on a publicly avail-able multi-lingual semantic parsing dataset ATIS and a newly constructed dataset. Experimentalresults show that our model outperforms state-of-the-art methods on both datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Semantic ParsingSentenceSimilar Papers 제목 키워드 기반
Multilingual Pretraining and Instruction Tuning Improve Cross-Lingual Knowledge Alignment, But Only Shallowly
Despite their strong ability to retrieve knowledge in English, current large language models show imbalance abilities in different languages. Two approaches are proposed to address this, i.e., multilingual pretraining an…
PreAlign: Boosting Cross-Lingual Transfer by Early Establishment of Multilingual Alignment
Large language models demonstrate reasonable multilingual abilities, despite predominantly English-centric pretraining. However, the spontaneous multilingual alignment in these models is shown to be weak, leading to unsa…
Cross-Lingual TransferLanguage ModelingLanguage ModellingZero-Shot Cross-Lingual TransferOn the limited utility of parallel data for learning shared multilingual representations
Shared multilingual representations are essential for cross-lingual tasks and knowledge transfer across languages. This study looks at the impact of parallel data, i.e. translated sentences, in pretraining as a signal to…
Can Machine Translation Bridge Multilingual Pretraining and Cross-lingual Transfer Learning?
Multilingual pretraining and fine-tuning have remarkably succeeded in various natural language processing tasks. Transferring representations from one language to another is especially crucial for cross-lingual learning.…
Cross-Lingual TransferMachine TranslationNatural Language UnderstandingRepresentation Learning+3Multilingual BERT Post-Pretraining Alignment
We propose a simple method to align multilingual contextual embeddings as a post-pretraining step for improved zero-shot cross-lingual transferability of the pretrained models. Using parallel data, our method aligns embe…
Contrastive LearningLanguage ModelingLanguage ModellingSentence+1