Switch-LSTMs for Multi-Criteria Chinese Word Segmentation
Multi-criteria Chinese word segmentation is a promising but challenging task, which exploits several different segmentation criteria and mines their common underlying knowledge. In this paper, we propose a flexible multi-criteria learning for Chinese word segmentation. Usually, a segmentation criterion could be decomposed into multiple sub-criteria, which are shareable with other segmentation criteria. The process of word segmentation is a routing among these sub-criteria. From this perspective, we present Switch-LSTMs to segment words, which consist of several long short-term memory neural networks (LSTM), and a switcher to automatically switch the routing among these LSTMs. With these auto-switched LSTMs, our model provides a more flexible solution for multi-criteria CWS, which is also easy to transfer the learned knowledge to new criteria. Experiments show that our model obtains significant improvements on eight corpora with heterogeneous segmentation criteria, compared to the previous method and single-criterion learning.
Code (0)
등록된 구현이 없습니다.
Tasks
Chinese Word SegmentationSegmentationSimilar Papers 제목 키워드 기반
Tackling Code-Switched NER: Participation of CMU
Named Entity Recognition plays a major role in several downstream applications in NLP. Though this task has been heavily studied in formal monolingual texts and also noisy texts like Twitter data, it is still an emerging…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER+1Investigating Self-Attention Network for Chinese Word Segmentation
Neural network has become the dominant method for Chinese word segmentation. Most existing models cast the task as sequence labeling, using BiLSTM-CRF for representing the input and making output predictions. Recently, a…
Chinese Word SegmentationSegmentationUnified Multi-Criteria Chinese Word Segmentation with BERT
Multi-Criteria Chinese Word Segmentation (MCCWS) aims at finding word boundaries in a Chinese sentence composed of continuous characters while multiple segmentation criteria exist. The unified framework has been widely u…
Chinese Word SegmentationLanguage ModelingLanguage ModellingMulti-Task Learning+2A Concise Model for Multi-Criteria Chinese Word Segmentation with Transformer Encoder
Multi-criteria Chinese word segmentation (MCCWS) aims to exploit the relations among the multiple heterogeneous segmentation criteria and further improve the performance of each single criterion. Previous work usually re…
Chinese Word SegmentationMulti-Task LearningSegmentationSurprisal Predicts Code-Switching in Chinese-English Bilingual Text
Why do bilinguals switch languages within a sentence? The present observational study asks whether word surprisal and word entropy predict code-switching in bilingual written conversation. We describe and model a new dat…
Sentence