Teaching Large Language Models an Unseen Language on the Fly
Existing large language models struggle to support numerous low-resource languages, particularly the extremely low-resource ones, for which there is minimal training data available for effective parameter updating. We thus investigate whether LLMs can learn a new language on the fly solely through prompting. To study this question, we collect a research suite for Zhuang, a language supported by no LLMs currently. We introduce DiPMT++, a framework for adapting LLMs to unseen languages by in-context learning. Using a dictionary and 5K parallel sentences only, DiPMT++ significantly enhances the performance of GPT-4 from 0 to 16 BLEU for Chinese-to-Zhuang translation and achieves 32 BLEU for Zhuang-to-Chinese translation. We also validate the effectiveness of our framework on Kalamang, another unseen language. Furthermore, we demonstrate the practical utility of DiPMT++ in aiding humans in translating completely unseen languages, which could contribute to the preservation of linguistic diversity.
Code (1)
Tasks
DiversityIn-Context LearningTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Characterizing tradeoffs between teaching via language and demonstrations in multi-agent systems
Humans teach others about the world through language and demonstration. When might one of these modalities be more effective than the other? In this work, we study the factors that modulate the effectiveness of language …
TeachBench: A Syllabus-Grounded Framework for Evaluating Teaching Ability in Large Language Models
Large language models (LLMs) show promise as teaching assistants, yet their teaching capability remains insufficiently evaluated. Existing benchmarks mainly focus on problem-solving or problem-level guidance, leaving kno…
It's All About In-Context Learning! Teaching Extremely Low-Resource Languages to LLMs
Extremely low-resource languages, especially those written in rare scripts, as shown in Figure 1, remain largely unsupported by large language models (LLMs). This is due in part to compounding factors such as the lack of…
parameter-efficient fine-tuningKU\_ai at MEDIQA 2019: Domain-specific Pre-training and Transfer Learning for Medical NLI
In this paper, we describe our system and results submitted for the Natural Language Inference (NLI) track of the MEDIQA 2019 Shared Task. As KU{\_}ai team, we used BERT as our baseline model and pre-processed the MedNLI…
De-identificationLanguage ModelingLanguage ModellingNatural Language Inference+1RuleBert: Teaching Soft Rules to Pre-trained Language Models
While pre-trained language models (PLMs) are the go-to solution to tackle many natural language processing problems, they are still very limited in their ability to capture and to use common-sense knowledge. In fact, eve…
Common Sense Reasoning