Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMs
We present two multilingual LLMs designed to embrace Europe's linguistic diversity by supporting all 24 official languages of the European Union. Trained on a dataset comprising around 60% non-English data and utilizing a custom multilingual tokenizer, our models address the limitations of existing LLMs that predominantly focus on English or a few high-resource languages. We detail the models' development principles, i.e., data composition, tokenizer optimization, and training methodologies. The models demonstrate competitive performance across multilingual benchmarks, as evidenced by their performance on European versions of ARC, HellaSwag, MMLU, and TruthfulQA.
Code (0)
등록된 구현이 없습니다.
Tasks
ARCDiversityHellaSwagMMLUTruthfulQAMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Security and Detectability Analysis of Unicode Text Watermarking Methods against Large Language Models
Securing digital text is becoming increasingly relevant due to the widespread use of large language models. Individuals' fear of losing control over data when it is being used to train such machine learning models or whe…
EuroAlpaca: Task-Preserving Localisation of Instruction Data for European Languages
Machine translation (MT) offers a scalable way to extend English instruction-tuning data to multiple languages, but it can distort task-critical constraints and required outputs, creating corrupted training examples and …
Instruction FollowingMachine TranslationInvestigating Multilingual Instruction-Tuning: Do Polyglot Models Demand for Multilingual Instructions?
The adaption of multilingual pre-trained LLMs into eloquent and helpful assistants is essential to facilitate their use across different language regions. In that spirit, we are the first to conduct an extensive study of…
Instruction FollowingLlama meets EU: Investigating the European Political Spectrum through the Lens of LLMs
Instruction-finetuned Large Language Models inherit clear political leanings that have been shown to influence downstream task performance. We expand this line of research beyond the two-party system in the US and audit …
Classifying German Language Proficiency Levels Using Large Language Models
Assessing language proficiency is essential for education, as it enables instruction tailored to learners needs. This paper investigates the use of Large Language Models (LLMs) for automatically classifying German texts …