Sparse Coding of Neural Word Embeddings for Multilingual Sequence Labeling
In this paper we propose and carefully evaluate a sequence labeling framework which solely utilizes sparse indicator features derived from dense distributed word representations. The proposed model obtains (near) state-of-the art performance for both part-of-speech tagging and named entity recognition for a variety of languages. Our model relies only on a few thousand sparse coding-derived features, without applying any modification of the word representations employed for the different tasks. The proposed model has favorable generalization properties as it retains over 89.8{\%} of its average POS tagging accuracy when trained at 1.2{\%} of the total available training data, i.e. 150 sentences per language.
Code (0)
등록된 구현이 없습니다.
Tasks
Feature Engineeringnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)Part-Of-Speech TaggingPOSPOS TaggingWord EmbeddingsSimilar Papers 제목 키워드 기반
Sparse Coding of Neural Word Embeddings for Multilingual Sequence Labeling
In this paper we propose and carefully evaluate a sequence labeling framework which solely utilizes sparse indicator features derived from dense distributed word representations. The proposed model obtains (near) state-o…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)Part-Of-Speech Tagging+3The Impact of Positional Encodings on Multilingual Compression
In order to preserve word-order information in a non-autoregressive setting, transformer architectures tend to include positional knowledge, by (for instance) adding positional encodings to token embeddings. Several modi…
Inductive BiasWord Equations: Inherently Interpretable Sparse Word Embeddingsthrough Sparse Coding
Word embeddings are a powerful natural language processing technique, but they are extremely difficult to interpret. To enable interpretable NLP models, we create vectors where each dimension is inherently interpretable.…
Word EmbeddingsWord Equations: Inherently Interpretable Sparse Word Embeddings through Sparse Coding
Word embeddings are a powerful natural language processing technique, but they are extremely difficult to interpret. To enable interpretable NLP models, we create vectors where each dimension is inherently interpretable.…
Word EmbeddingsSequence Tagging with Contextual and Non-Contextual Subword Representations: A Multilingual Evaluation
Pretrained contextual and non-contextual subword embeddings have become available in over 250 languages, allowing massively multilingual NLP. However, while there is no dearth of pretrained embeddings, the distinct lack …
Multilingual Named Entity RecognitionMultilingual NLPnamed-entity-recognitionNamed Entity Recognition+2