Naive Bayes and BiLSTM Ensemble for Discriminating between Mainland and Taiwan Variation of Mandarin Chinese
Automatic dialect identification is a more challengingctask than language identification, as it requires the ability to discriminate between varieties of one language. In this paper, we propose an ensemble based system, which combines traditional machine learning models trained on bag of n-gram fetures, with deep learning models trained on word embeddings, to solve the Discriminating between Mainland and Taiwan Variation of Mandarin Chinese (DMT) shared task at VarDial 2019. Our experiments show that a character bigram-trigram combination based Naive Bayes is a very strong model for identifying varieties of Mandarin Chinense. Through further ensemble of Navie Bayes and BiLSTM, our system (team: itsalexyang) achived an macro-averaged F1 score of 0.8530 and 0.8687 in two tracks.
Code (0)
등록된 구현이 없습니다.
Tasks
Dialect IdentificationLanguage IdentificationWord EmbeddingsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Ensemble Methods to Distinguish Mainland and Taiwan Chinese
This paper describes the IUCL system at VarDial 2019 evaluation campaign for the task of discriminating between Mainland and Taiwan variation of mandarin Chinese. We first build several base classifiers, including a Naiv…
Word EmbeddingsDiscriminating between Mandarin Chinese and Swiss-German varieties using adaptive language models
This paper describes the language identification systems used by the SUKI team in the Discriminating between the Mainland and Taiwan variation of Mandarin Chinese (DMT) and the German Dialect Identification (GDI) shared …
Dialect IdentificationLanguage IdentificationLanguage ModelingLanguage ModellingComparing Two Basic Methods for Discriminating Between Similar Languages and Varieties
This article describes the systems submitted by the Citius{\_}Ixa{\_}Imaxin team to the Discriminating Similar Languages Shared Task 2016. The systems are based on two different strategies: classification with ranked dic…
Automatic Speech Recognition (ASR)General ClassificationLanguage IdentificationSpeech Recognition+1When Simple n-gram Models Outperform Syntactic Approaches: Discriminating between Dutch and Flemish
In this paper we present the results of our participation in the Discriminating between Dutch and Flemish in Subtitles VarDial 2018 shared task. We try techniques proven to work well for discriminating between language v…
RELIANCE: Reliable Ensemble Learning for Information and News Credibility Evaluation
In the era of information proliferation, discerning the credibility of news content poses an ever-growing challenge. This paper introduces RELIANCE, a pioneering ensemble learning system designed for robust information a…
Ensemble Learning