Exploring Multilingual Probing in Large Language Models: A Cross-Language Analysis
Probing techniques for large language models (LLMs) have primarily focused on English, overlooking the vast majority of the world's languages. In this paper, we extend these probing methods to a multilingual context, investigating the behaviors of LLMs across diverse languages. We conduct experiments on several open-source LLM models, analyzing probing accuracy, trends across layers, and similarities between probing vectors for multiple languages. Our key findings reveal: (1) a consistent performance gap between high-resource and low-resource languages, with high-resource languages achieving significantly higher probing accuracy; (2) divergent layer-wise accuracy trends, where high-resource languages show substantial improvement in deeper layers similar to English; and (3) higher representational similarities among high-resource languages, with low-resource languages demonstrating lower similarities both among themselves and with high-resource languages. These results highlight significant disparities in LLMs' multilingual capabilities and emphasize the need for improved modeling of low-resource languages.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Morph Call: Probing Morphosyntactic Content of Multilingual Transformers
The outstanding performance of transformer-based language models on a great variety of NLP and NLU tasks has stimulated interest in exploring their inner workings. Recent research has focused primarily on higher-level an…
Common Sense ReasoningMORPHPOSPOS Tagging+2LINSPECTOR: Multilingual Probing Tasks for Word Representations
Despite an ever growing number of word representation models introduced for a large number of languages, there is a lack of a standardized technique to provide insights into what is captured by these models. Such insight…
Dependency Parsingnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+6Exploring Cross-lingual Latent Transplantation: Mutual Opportunities and Open Challenges
Current large language models (LLMs) often exhibit imbalances in multilingual capabilities and cultural adaptability, largely attributed to their English-centric pre-training data. In this paper, we introduce and investi…
Multilingual Probing of Deep Pre-Trained Contextual Encoders
Encoders that generate representations based on context have, in recent years, benefited from adaptations that allow for pre-training on large text corpora. Earlier work on evaluating fixed-length sentence representation…
DiagnosticSentenceProbing LLMs for Multilingual Discourse Generalization Through a Unified Label Set
Discourse understanding is essential for many NLP tasks, yet most existing work remains constrained by framework-dependent discourse representations. This work investigates whether large language models (LLMs) capture di…
RelationRelation Classification