Automatic Identification of Closely-related Indian Languages: Resources and Experiments
In this paper, we discuss an attempt to develop an automatic language identification system for 5 closely-related Indo-Aryan languages of India, Awadhi, Bhojpuri, Braj, Hindi and Magahi. We have compiled a comparable corpora of varying length for these languages from various resources. We discuss the method of creation of these corpora in detail. Using these corpora, a language identification system was developed, which currently gives state of the art accuracy of 96.48\%. We also used these corpora to study the similarity between the 5 languages at the lexical level, which is the first data-based study of the extent of closeness of these languages.
Code (0)
등록된 구현이 없습니다.
Tasks
Language IdentificationSimilar Papers 제목 키워드 기반
Utilizing Wordnets for Cognate Detection among Indian Languages
Automatic Cognate Detection (ACD) is a challenging task which has been utilized to help NLP applications like Machine Translation, Information Retrieval and Computational Phylogenetics. Unidentified cognate pairs can pos…
Information RetrievalMachine TranslationRetrievalUnsupervised Deep Language and Dialect Identification for Short Texts
Automatic Language Identification (LI) or Dialect Identification (DI) of short texts of closely related languages or dialects, is one of the primary steps in many natural language processing pipelines. Language identific…
Dialect IdentificationLanguage IdentificationSentenceSentence EmbeddingsExploiting Spectral Augmentation for Code-Switched Spoken Language Identification
Spoken language Identification (LID) systems are needed to identify the language(s) present in a given audio sample, and typically could be the first step in many speech processing related tasks such as automatic speech …
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Language Identificationspeech-recognition+2Discriminating Between Similar Nordic Languages
Automatic language identification is a challenging problem. Discriminating between closely related languages is especially difficult. This paper presents a machine learning approach for automatic language identification …
BIG-bench Machine LearningLanguage IdentificationScript-Agnostic Language Identification
Language identification is used as the first step in many data collection and crawling efforts because it allows us to sort online text into language-specific buckets. However, many modern languages, such as Konkani, Kas…
Language Identification