paper-with-me

홈 › Papers

Bridging the Language Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs

2023-05-28 · Somnath Kumar, Vaibhav Balloli, Mercy Ranjit, Kabir Ahuja, Sunayana Sitaram, Kalika Bali, Tanuja Ganu, Akshay Nambi

Large language models (LLMs) have revolutionized various domains but still struggle with non-Latin scripts and low-resource languages. This paper addresses the critical challenge of improving multilingual performance without extensive fine-tuning. We introduce a novel dynamic learning approach that optimizes prompt strategy, embedding model, and LLM per query at runtime. By adapting configurations dynamically, our method achieves significant improvements over static, best and random baselines. It operates efficiently in both offline and online settings, generalizing seamlessly across new languages and datasets. Leveraging Retrieval-Augmented Generation (RAG) with state-of-the-art multilingual embeddings, we achieve superior task performance across diverse linguistic contexts. Through systematic investigation and evaluation across 18 diverse languages using popular question-answering (QA) datasets we show our approach results in 10-15% improvements in multilingual performance over pre-trained models and 4x gains compared to fine-tuned, language-specific models.

📄 PDF Abstract BibTeX arXiv:2305.17740

Code (0)

등록된 구현이 없습니다.

Tasks

Question AnsweringRAGRetrievalRetrieval-augmented Generation

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음
Attention 설명 없음
Cosine Annealing Cosine Annealing is a type of learning rate schedule that has the effect of starting with a large learning rate that is relatively rapidly decreased to a minimum value before…
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Similar Papers 제목 키워드 기반

Bridging the Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs

2024-05-28 · Somnath Kumar, Vaibhav Balloli, Mercy Ranjit, Kabir Ahuja 외

Large language models (LLMs) are at the forefront of transforming numerous domains globally. However, their inclusivity and effectiveness remain limited for non-Latin scripts and low-resource languages. This paper tackle…

Question AnsweringRAGRetrieval-augmented Generation

Blessing of Multilinguality: A Systematic Analysis of Multilingual In-Context Learning

2025-02-17 · Yilei Tu, Andrew Xue, Freda Shi

While multilingual large language models generally perform adequately, and sometimes even rival English performance on high-resource languages (HRLs), they often significantly underperform on low-resource languages (LRLs…

Cross-Lingual TransferIn-Context Learning

Marco-LLM: Bridging Languages via Massive Multilingual Training for Cross-Lingual Enhancement

2024-12-05 · Lingfeng Ming, Bo Zeng, Chenyang Lyu, Tianqi Shi 외

Large Language Models (LLMs) have achieved remarkable progress in recent years; however, their excellent performance is still largely limited to major world languages, primarily English. Many LLMs continue to face challe…

BelebeleMachine Translation

Multilingual Conversational AI for Financial Assistance: Bridging Language Barriers in Indian FinTech

2025-12-01 · Bharatdeep Hazarika, Arya Suneesh, Prasanna Devadiga, Pawan Kumar Rajpoot 외 arxiv

India's linguistic diversity presents both opportunities and challenges for fintech platforms. While the country has 31 major languages and over 100 minor ones, only 10\% of the population understands English, creating b…

Response Generation

LangBridge: Multilingual Reasoning Without Multilingual Supervision

2024-01-19 · Dongkeun Yoon, Joel Jang, Sungdong Kim, Seungone Kim 외

We introduce LangBridge, a zero-shot approach to adapt language models for multilingual reasoning tasks without multilingual supervision. LangBridge operates by bridging two models, each specialized in different aspects:…

Code CompletionLogical ReasoningMathematical Reasoning