홈 › Papers

FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Data

2024-08-12 · Haoran Sun, Renren Jin, Shaoyang Xu, Leiyu Pan, Supryadi, Menglong Cui, Jiangcun Du, Yikun Lei, Lei Yang, Ling Shi, Juesi Xiao, Shaolin Zhu, Deyi Xiong

Large language models (LLMs) have demonstrated prowess in a wide range of tasks. However, many LLMs exhibit significant performance discrepancies between high- and low-resource languages. To mitigate this challenge, we present FuxiTranyu, an open-source multilingual LLM, which is designed to satisfy the need of the research community for balanced and high-performing multilingual capabilities. The base model, FuxiTranyu-8B, features 8 billion parameters and is trained from scratch on meticulously balanced multilingual data that contains 600 billion tokens covering 43 natural languages and 16 programming languages. We also develop two instruction-tuned models: FuxiTranyu-8B-SFT which is fine-tuned on a diverse multilingual instruction dataset, and FuxiTranyu-8B-DPO which is further refined with DPO on a preference dataset for enhanced alignment ability. Extensive experiments on a wide range of multilingual benchmarks demonstrate the competitive performance of FuxiTranyu against existing multilingual LLMs, e.g., BLOOM-7B, PolyLM-13B, and Mistral-7B-Instruct. Both neuron and representation interpretability analyses reveal that FuxiTranyu achieves consistent multilingual representations across languages. To promote further research into multilingual LLMs, we release both the base and instruction-tuned FuxiTranyu models together with 58 pre-training checkpoints at HuggingFace (see https://huggingface.co/TJUNLP/FuxiTranyu-8B) and Github (see https://github.com/tjunlp-lab/FuxiTranyu).

📄 PDF Abstract BibTeX arXiv:2408.06273

Code (1)

tjunlp-lab/fuxitranyu 공식 구현

Tasks

Language ModelingLanguage ModellingLarge Language Model

Methods 이 논문이 사용한 방법론

DPO 설명 없음

BASE 설명 없음

Tiny Aya: Bridging Scale and Multilingual Depth

2026-03-12 · Alejandro R. Salamanca, Diana Abagyan, Daniel D'souza, Ammar Khairi 외 arxiv

Tiny Aya redefines what a small multilingual language model can achieve. Trained on 70 languages and refined through region-aware posttraining, it delivers state-of-the-art in translation quality, strong multilingual und…

AlignX: Advancing Multilingual Large Language Models with Multilingual Representation Alignment

2025-09-29 · Mengyu Bu, Shaolei Zhang, Zhongjun He, Hua Wu 외 arxiv

Multilingual large language models (LLMs) possess impressive multilingual understanding and generation capabilities. However, their performance and cross-lingual alignment often lag for non-dominant languages. A common s…

Language on Demand, Knowledge at Core: Composing LLMs with Encoder-Decoder Translation Models for Extensible Multilinguality

2026-03-18 · Mengyu Bu, Yang Feng arxiv

Large language models (LLMs) exhibit strong general intelligence, yet their multilingual performance remains highly imbalanced. Although LLMs encode substantial cross-lingual knowledge in a unified semantic space, they o…

General Knowledge

Polyglot-Lion: Efficient Multilingual ASR for Singapore via Balanced Fine-Tuning of Qwen3-ASR

2026-03-17 · Quy-Anh Dang, Chris Ngo arxiv

We present Polyglot-Lion, a family of compact multilingual automatic speech recognition (ASR) models tailored for the linguistic landscape of Singapore, covering English, Mandarin, Tamil, and Malay. Our models are obtain…

Speech Recognition

Pruning Multilingual Large Language Models for Multilingual Inference

2024-09-25 · Hwichan Kim, Jun Suzuki, Tosho Hirasawa, Mamoru Komachi

Multilingual large language models (MLLMs), trained on multilingual balanced data, demonstrate better zero-shot learning performance in non-English languages compared to large language models trained on English-dominant …

Translation Zero-Shot Learning