LSTM Autoencoders for Dialect Analysis
Computational approaches for dialectometry employed Levenshtein distance to compute an aggregate similarity between two dialects belonging to a single language group. In this paper, we apply a sequence-to-sequence autoencoder to learn a deep representation for words that can be used for meaningful comparison across dialects. In contrast to the alignment-based methods, our method does not require explicit alignments. We apply our architectures to three different datasets and show that the learned representations indicate highly similar results with the analyses based on Levenshtein distance and capture the traditional dialectal differences shown by dialectologists.
Code (0)
등록된 구현이 없습니다.
Tasks
Dimensionality ReductionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Pre-trained Models or Feature Engineering: The Case of Dialectal Arabic
The usage of social media platforms has resulted in the proliferation of work on Arabic Natural Language Processing (ANLP), including the development of resources. There is also an increased interest in processing Arabic…
Dialect IdentificationFeature EngineeringSentiment AnalysisWord EmbeddingsDiscrimination between Similar Languages, Varieties and Dialects using CNN- and LSTM-based Deep Neural Networks
In this paper, we describe a system (CGLI) for discriminating similar languages, varieties and dialects using convolutional neural networks (CNNs) and long short-term memory (LSTM) neural networks. We have participated i…
Dialect IdentificationInformation RetrievalLanguage IdentificationMachine Translation+2A Deep Learning Approach for Similar Languages, Varieties and Dialects
Deep learning mechanisms are prevailing approaches in recent days for the various tasks in natural language processing, speech recognition, image processing and many others. To leverage this we use deep learning based me…
Deep LearningDialect Identificationspeech-recognitionSpeech RecognitionSpeech Representations and Phoneme Classification for Preserving the Endangered Language of Ladin
A vast majority of the world's 7,000 spoken languages are predicted to become extinct within this century, including the endangered language of Ladin from the Italian Alps. Linguists who work to preserve a language's pho…
Arabic Multi-Dialect Segmentation: bi-LSTM-CRF vs. SVM
Arabic word segmentation is essential for a variety of NLP applications such as machine translation and information retrieval. Segmentation entails breaking words into their constituent stems, affixes and clitics. In thi…
Domain AdaptationInformation RetrievalMachine TranslationRetrieval+3