paper-with-me

홈 › Papers

Switch-LSTMs for Multi-Criteria Chinese Word Segmentation

2018-12-19 · Jingjing Gong, Xinchi Chen, Tao Gui, Xipeng Qiu

Multi-criteria Chinese word segmentation is a promising but challenging task, which exploits several different segmentation criteria and mines their common underlying knowledge. In this paper, we propose a flexible multi-criteria learning for Chinese word segmentation. Usually, a segmentation criterion could be decomposed into multiple sub-criteria, which are shareable with other segmentation criteria. The process of word segmentation is a routing among these sub-criteria. From this perspective, we present Switch-LSTMs to segment words, which consist of several long short-term memory neural networks (LSTM), and a switcher to automatically switch the routing among these LSTMs. With these auto-switched LSTMs, our model provides a more flexible solution for multi-criteria CWS, which is also easy to transfer the learned knowledge to new criteria. Experiments show that our model obtains significant improvements on eight corpora with heterogeneous segmentation criteria, compared to the previous method and single-criterion learning.

📄 PDF Abstract BibTeX arXiv:1812.08033

Code (0)

등록된 구현이 없습니다.

Tasks

Chinese Word SegmentationSegmentation

Similar Papers 제목 키워드 기반

Tackling Code-Switched NER: Participation of CMU

2018-07-01 · WS 2018 7 · Parvathy Geetha, Ch, Khyathi u, Alan W. black

Named Entity Recognition plays a major role in several downstream applications in NLP. Though this task has been heavily studied in formal monolingual texts and also noisy texts like Twitter data, it is still an emerging…

named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER+1

Investigating Self-Attention Network for Chinese Word Segmentation

2019-07-26 · Leilei Gan, Yue Zhang

Neural network has become the dominant method for Chinese word segmentation. Most existing models cast the task as sequence labeling, using BiLSTM-CRF for representing the input and making output predictions. Recently, a…

Chinese Word SegmentationSegmentation

Unified Multi-Criteria Chinese Word Segmentation with BERT

2020-04-13 · Zhen Ke, Liang Shi, Erli Meng, Bin Wang 외

Multi-Criteria Chinese Word Segmentation (MCCWS) aims at finding word boundaries in a Chinese sentence composed of continuous characters while multiple segmentation criteria exist. The unified framework has been widely u…

Chinese Word SegmentationLanguage ModelingLanguage ModellingMulti-Task Learning+2

A Concise Model for Multi-Criteria Chinese Word Segmentation with Transformer Encoder

2019-06-28 · Findings of the Association for Computational Linguistics 2020 · Xipeng Qiu, Hengzhi Pei, Hang Yan, Xuanjing Huang

Multi-criteria Chinese word segmentation (MCCWS) aims to exploit the relations among the multiple heterogeneous segmentation criteria and further improve the performance of each single criterion. Previous work usually re…

Chinese Word SegmentationMulti-Task LearningSegmentation

Surprisal Predicts Code-Switching in Chinese-English Bilingual Text

2020-11-01 · EMNLP 2020 11 · Jes{\'u}s Calvillo, Le Fang, Jeremy Cole, David Reitter

Why do bilinguals switch languages within a sentence? The present observational study asks whether word surprisal and word entropy predict code-switching in bilingual written conversation. We describe and model a new dat…

Sentence