Language Identification of Bengali-English Code-Mixed data using Character & Phonetic based LSTM Models

2018-03-10 · Soumil Mandal, Sourya Dipta Das, Dipankar Das

Language identification of social media text still remains a challenging task due to properties like code-mixing and inconsistent phonetic transliterations. In this paper, we present a supervised learning approach for language identification at the word level of low resource Bengali-English code-mixed data taken from social media. We employ two methods of word encoding, namely character based and root phone based to train our deep LSTM models. Utilizing these two models we created two ensemble models using stacking and threshold technique which gave 91.78% and 92.35% accuracies respectively on our testing data.

📄 PDF Abstract BibTeX arXiv:1803.03859

Code (0)

등록된 구현이 없습니다.

Tasks

Language Identification

Methods 이 논문이 사용한 방법론

Sigmoid Activation 설명 없음

Tanh Activation 설명 없음

LSTM An LSTM is a type of recurrent neural network that addresses the vanishing gradient problem in vanilla…

Development of POS tagger for English-Bengali Code-Mixed data

2020-07-29 · ICON 2019 12 · Tathagata Raha, Sainik Kumar Mahata, Dipankar Das, Sivaji Bandyopadhyay

Code-mixed texts are widespread nowadays due to the advent of social media. Since these texts combine two languages to formulate a sentence, it gives rise to various research problems related to Natural Language Processi…

POS Sentence TAG

JU_KS@SAIL_CodeMixed-2017: Sentiment Analysis for Indian Code Mixed Social Media Texts

2018-02-15 · Kamal Sarkar

This paper reports about our work in the NLP Tool Contest @ICON-2017, shared task on Sentiment Analysis for Indian Languages (SAIL) (code mixed). To implement our system, we have used a machine learning algo-rithm called…

Position Sentiment Analysis

BnSentMix: A Diverse Bengali-English Code-Mixed Dataset for Sentiment Analysis

2024-08-16 · Sadia Alam, Md Farhan Ishmam, Navid Hasin Alvee, Md Shahnewaz Siddique 외

The widespread availability of code-mixed data can provide valuable insights into low-resource languages like Bengali, which have limited datasets. Sentiment analysis has been a fundamental text classification task acros…

Diversity Sentiment Analysis Sentiment Classification text-classification+1

Analyzing Roles of Classifiers and Code-Mixed factors for Sentiment Identification

2018-01-08 · Soumil Mandal, Dipankar Das

Multilingual speakers often switch between languages to express themselves on social communication platforms. Sometimes, the original script of the language is preserved, while using a common script for all the languages…

Leveraging Sub Label Dependencies in Code Mixed Indian Languages for Part-Of-Speech Tagging using Conditional Random Fields.

2022-06-01 · WILDRE (LREC) 2022 6 · Akash Kumar Gautam

Code-mixed text sequences often lead to challenges in the task of correct identification of Part-Of-Speech tags. However, lexical dependencies created while alternating between multiple languages can be leveraged to impr…

Part-Of-Speech Tagging