Transformer-based Model for Word Level Language Identification in Code-mixed Kannada-English Texts
Using code-mixed data in natural language processing (NLP) research currently gets a lot of attention. Language identification of social media code-mixed text has been an interesting problem of study in recent years due to the advancement and influences of social media in communication. This paper presents the Instituto Polit\'ecnico Nacional, Centro de Investigaci\'on en Computaci\'on (CIC) team's system description paper for the CoLI-Kanglish shared task at ICON2022. In this paper, we propose the use of a Transformer based model for word-level language identification in code-mixed Kannada English texts. The proposed model on the CoLI-Kenglish dataset achieves a weighted F1-score of 0.84 and a macro F1-score of 0.61.
Code (0)
등록된 구현이 없습니다.
Tasks
Language IdentificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Word-level Language Identification Using Subword Embeddings for Code-mixed Bangla-English Social Media Data
This paper reports work on building a word-level language identification (LID) model for code-mixed Bangla-English social media data using subword embeddings, with an ultimate goal of using this LID module as the first s…
Language IdentificationPOSCombining ResNet and Transformer for Chinese Grammatical Error Diagnosis
Grammatical error diagnosis is an important task in natural language processing. This paper introduces our system at NLPTEA-2020 Task: Chinese Grammatical Error Diagnosis (CGED). CGED aims to diagnose four types of gramm…
PositionSubword-Level Language Identification for Intra-Word Code-Switching
Language identification for code-switching (CS), the phenomenon of alternating between two or more languages in conversations, has traditionally been approached under the assumption of a single language per token. Howeve…
Language IdentificationHierarchical Character-Word Models for Language Identification
Social media messages' brevity and unconventional spelling pose a challenge to language identification. We introduce a hierarchical model that learns character and contextualized word-level representations for language i…
Language IdentificationTransliteration for Low-Resource Code-Switching Texts: Building an Automatic Cyrillic-to-Latin Converter for Tatar
We introduce a Cyrillic-to-Latin transliterator for the Tatar language based on subword-level language identification. The transliteration is a challenging task due to the following two reasons. First, because modern Tat…
Language IdentificationTransliteration