Syllable-aware Neural Language Models: A Failure to Beat Character-aware Ones
Syllabification does not seem to improve word-level RNN language modeling quality when compared to character-based segmentation. However, our best syllable-aware language model, achieving performance comparable to the competitive character-aware model, has 18%-33% fewer parameters and is trained 1.2-2.2 times faster.
Code (1)
Tasks
Language ModelingLanguage ModellingSegmentationSimilar Papers 제목 키워드 기반
Reusing Weights in Subword-aware Neural Language Models
We propose several ways of reusing subword embeddings and other weights in subword-aware neural language models. The proposed techniques do not benefit a competitive character-aware model, but some of them improve the pe…
Introducing Syllable Tokenization for Low-resource Languages: A Case Study with Swahili
Many attempts have been made in multilingual NLP to ensure that pre-trained language models, such as mBERT or GPT2 get better and become applicable to low-resource languages. To achieve multilingualism for pre-trained la…
Multilingual NLPText GenerationWord EmbeddingsRelationships between Keywords and Strong Beats in Lyrical Music
Artificial Intelligence (AI) song generation has emerged as a popular topic, yet the focus on exploring the latent correlations between specific lyrical and rhythmic features remains limited. In contrast, this pilot stud…
RhythmRevisiting Neural Language Modelling with Syllables
Language modelling is regularly analysed at word, subword or character units, but syllables are seldom used. Syllables provide shorter sequences than characters, they can be extracted with rules, and their segmentation t…
Language ModellingRevisiting Syllables in Language Modelling and their Application on Low-Resource Machine Translation
Language modelling and machine translation tasks mostly use subword or character inputs, but syllables are seldom used. Syllables provide shorter sequences than characters, require less-specialised extracting rules than …
Language ModellingMachine TranslationSegmentationTranslation