Revisiting Neural Language Modelling with Syllables
Language modelling is regularly analysed at word, subword or character units, but syllables are seldom used. Syllables provide shorter sequences than characters, they can be extracted with rules, and their segmentation typically requires less specialised effort than identifying morphemes. We reconsider syllables for an open-vocabulary generation task in 20 languages. We use rule-based syllabification methods for five languages and address the rest with a hyphenation tool, which behaviour as syllable proxy is validated. With a comparable perplexity, we show that syllables outperform characters, annotated morphemes and unsupervised subwords. Finally, we also study the overlapping of syllables concerning other subword pieces and discuss some limitations and opportunities.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModellingSimilar Papers 제목 키워드 기반
Revisiting Syllables in Language Modelling and their Application on Low-Resource Machine Translation
Language modelling and machine translation tasks mostly use subword or character inputs, but syllables are seldom used. Syllables provide shorter sequences than characters, require less-specialised extracting rules than …
Language ModellingMachine TranslationSegmentationTranslationAnalyzing Well-Formedness of Syllables in Japanese Sign Language
This paper tackles a problem of analyzing the well-formedness of syllables in Japanese Sign Language (JSL). We formulate the problem as a classification problem that classifies syllables into well-formed or ill-formed. W…
Active LearningSpeech and Language Resources for LVCSR of Russian
A syllable-based language model reduces the lexicon size by hundreds of times. It is especially beneficial in case of highly inflective languages like Russian due to the abundance of word forms according to various gramm…
Language ModelingLanguage ModellingSentenceLearning variable length units for SMT between related languages via Byte Pair Encoding
We explore the use of segments learnt using Byte Pair Encoding (referred to as BPE units) as basic units for statistical machine translation between related languages and compare it with orthographic syllables, which are…
Machine TranslationTranslationTowards Language Modelling in the Speech Domain Using Sub-word Linguistic Units
Language models (LMs) for text data have been studied extensively for their usefulness in language generation and other downstream tasks. However, language modelling purely in the speech domain is still a relatively unex…
Language ModellingText Generation