Utilizing Subword Entities in Character-Level Sequence-to-Sequence Lemmatization Models
In this paper we present a character-level sequence-to-sequence lemmatization model, utilizing several subword features in multiple configurations. In addition to generic n-gram embeddings (using FastText), we experiment with concatenative (stems) and templatic (roots and patterns) morphological subwords. We present several architectures that embed these features directly at the encoder side, or learn them jointly at the decoder side with a multitask learning architecture. The results indicate that using the generic n-gram embeddings (through FastText) outperform the other linguistically-driven subwords. We use Modern Standard Arabic and Egyptian Arabic as test cases, with up to 22{\%} and 13{\%} relative error reduction, respectively, from a strong baseline. An error analysis shows that our best system is even able to handle word/lemma pairs that are both unseen in the training data.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderLEMMALemmatizationSimilar Papers 제목 키워드 기반
Supersense Tagging with a Combination of Character, Subword, and Word-level Representations
Recently, there has been increased interest in utilizing characters or subwords for natural language processing (NLP) tasks. However, the effect of utilizing character, subword, and word-level information simultaneously …
Entity AlignmentLanguage ModelingLanguage ModellingMachine Translation+2Local Byte Fusion for Neural Machine Translation
Subword tokenization schemes are the dominant technique used in current NLP models. However, such schemes can be rigid and tokenizers built on one corpus do not adapt well to other parallel corpora. It has also been obse…
Cross-Lingual TransferDomain AdaptationMachine TranslationTranslation+1Fast and Accurate Transformer-based Translation with Character-Level Encoding and Subword-Level Decoding
The Transformer translation model is fast to train and achieves state-of-the-art results for various translation tasks. However, unknown input words at test time remain a challenge for the Transformer, especially when …
DecoderTranslationBeyond Characters: Subword-level Morpheme Segmentation
This paper presents DeepSPIN’s submissions to the SIGMORPHON 2022 Shared Task on Morpheme Segmentation. We make three submissions, all to the word-level subtask. First, we show that entmax-based sparse sequence-tosequenc…
Language ModelingLanguage ModellingMorpheme SegmentaitonSegmentation+12kenize: Tying Subword Sequences for Chinese Script Conversion
Simplified Chinese to Traditional Chinese character conversion is a common preprocessing step in Chinese NLP. Despite this, current approaches have poor performance because they do not take into account that a simplified…
General ClassificationTopic Classification