Do Multilingual Language Models Think Better in English?
Translate-test is a popular technique to improve the performance of multilingual language models. This approach works by translating the input into English using an external machine translation system, and running inference over the translated input. However, these improvements can be attributed to the use of a separate translation system, which is typically trained on large amounts of parallel data not seen by the language model. In this work, we introduce a new approach called self-translate, which overcomes the need of an external translation system by leveraging the few-shot translation capabilities of multilingual language models. Experiments over 5 tasks show that self-translate consistently outperforms direct inference, demonstrating that language models are unable to leverage their full multilingual potential when prompted in non-English languages. Our code is available at https://github.com/juletx/self-translate.
Code (1)
Tasks
Common Sense ReasoningCross-Lingual Natural Language InferenceCross-Lingual Paraphrase IdentificationLanguage ModelingLanguage ModellingMachine TranslationMath Word Problem SolvingNatural Language InferenceParaphrase IdentificationTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Could Thinking Multilingually Empower LLM Reasoning?
Previous work indicates that large language models exhibit a significant "English bias", i.e. they often perform better when tasks are presented in English. Interestingly, we have observed that using certain other langua…
Answer SelectionDictionary Insertion Prompting for Multilingual Reasoning on Multilingual Large Language Models
As current training data for Large Language Models (LLMs) are dominated by English corpus, they are English-centric and they present impressive performance on English reasoning tasks.\footnote{This paper primarily studie…
GSM8KMathConsiderations for Multilingual Wikipedia Research
English Wikipedia has long been an important data source for much research and natural language machine learning modeling. The growth of non-English language editions of Wikipedia, greater computational resources, and ca…
Language of Thought Shapes Output Diversity in Large Language Models
Output diversity is crucial for Large Language Models as it underpins pluralism and creativity. In this work, we reveal that controlling the language used during model thinking-the language of thought-provides a novel an…
Think Natively: Unlocking Multilingual Reasoning with Consistency-Enhanced Reinforcement Learning
Large Reasoning Models (LRMs) have achieved remarkable performance on complex reasoning tasks by adopting the ``think-then-answer'' paradigm, which enhances both accuracy and interpretability. However, current LRMs exhib…
Reinforcement Learning