Word-like character n-gram embedding
We propose a new word embedding method called \textit{word-like character} n\textit{-gram embedding}, which learns distributed representations of words by embedding word-like character n-grams. Our method is an extension of recently proposed \textit{segmentation-free word embedding}, which directly embeds frequent character n-grams from a raw corpus. However, its n-gram vocabulary tends to contain too many non-word n-grams. We solved this problem by introducing an idea of \textit{expected word frequency}. Compared to the previously proposed methods, our method can embed more words, along with the words that are not included in a given basic word dictionary. Since our method does not rely on word segmentation with rich word dictionaries, it is especially effective when the text in the corpus is in unsegmented language and contains many neologisms and informal words (e.g., Chinese SNS dataset). Our experimental results on Sina Weibo (a Chinese microblog service) and Twitter show that the proposed method can embed more words and improve the performance of downstream tasks.
Code (1)
Tasks
SegmentationWord EmbeddingsSimilar Papers 제목 키워드 기반
Improving Chinese Segmentation-free Word Embedding With Unsupervised Association Measure
Recent work on segmentation-free word embedding(sembei) developed a new pipeline of word embedding for unsegmentated language while avoiding segmentation as a preprocessing step. However, too many noisy n-grams existing …
SegmentationvalidCharagram: Embedding Words and Sentences via Character n-grams
We present Charagram embeddings, a simple approach for learning character-based compositional models to embed textual sequences. A word or sentence is represented using a character n-gram count vector, followed by a sing…
Part-Of-Speech TaggingSentenceSentence SimilarityWord SimilarityClassification of Micro-Texts Using Sub-Word Embeddings
Extracting features and writing styles from short text messages is always a challenge. Short messages, like tweets, do not have enough data to perform statistical authorship attribution. Besides, the vocabulary used in t…
Authorship AttributionClassificationGeneral ClassificationWord EmbeddingsCharacter n-gram Embeddings to Improve RNN Language Models
This paper proposes a novel Recurrent Neural Network (RNN) language model that takes advantage of character information. We focus on character n-grams based on research in the field of word embedding construction (Wietin…
Headline GenerationLanguage ModelingLanguage ModellingMachine Translation+2Data-Driven Mitigation of Adversarial Text Perturbation
Social networks have become an indispensable part of our lives, with billions of people producing ever-increasing amounts of text. At such scales, content policies and their enforcement become paramount. To automate mode…
Adversarial TextClassificationRobust classificationWord Embeddings