BERT Meets Chinese Word Segmentation
Chinese word segmentation (CWS) is a fundamental task for Chinese language understanding. Recently, neural network-based models have attained superior performance in solving the in-domain CWS task. Last year, Bidirectional Encoder Representation from Transformers (BERT), a new language representation model, has been proposed as a backbone model for many natural language tasks and redefined the corresponding performance. The excellent performance of BERT motivates us to apply it to solve the CWS task. By conducting intensive experiments in the benchmark datasets from the second International Chinese Word Segmentation Bake-off, we obtain several keen observations. BERT can slightly improve the performance even when the datasets contain the issue of labeling inconsistency. When applying sufficiently learned features, Softmax, a simpler classifier, can attain the same performance as that of a more complicated classifier, e.g., Conditional Random Field (CRF). The performance of BERT usually increases as the model size increases. The features extracted by BERT can be also applied as good candidates for other neural network models.
Code (0)
등록된 구현이 없습니다.
Tasks
Chinese Word SegmentationSegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Character, Word, or Both? Revisiting the Segmentation Granularity for Chinese Pre-trained Language Models
Pretrained language models (PLMs) have shown marvelous improvements across various NLP tasks. Most Chinese PLMs simply treat an input text as a sequence of characters, and completely ignore word information. Although Who…
A Graph-based Model for Joint Chinese Word Segmentation and Dependency Parsing
Chinese word segmentation and dependency parsing are two fundamental tasks for Chinese natural language processing. The dependency parsing is defined on word-level. Therefore word segmentation is the precondition of depe…
Chinese Word SegmentationDependency ParsingFeature EngineeringLanguage Modeling+2Unified Multi-Criteria Chinese Word Segmentation with BERT
Multi-Criteria Chinese Word Segmentation (MCCWS) aims at finding word boundaries in a Chinese sentence composed of continuous characters while multiple segmentation criteria exist. The unified framework has been widely u…
Chinese Word SegmentationLanguage ModelingLanguage ModellingMulti-Task Learning+2Unsupervised Chinese Word Segmentation with BERT Oriented Probing and Transformation
Word Segmentation is a fundamental step for understanding Chinese language. Previous neural approaches for unsupervised Chinese Word Segmentation (CWS) only exploits shallow semantic information, which can miss important…
Chinese Word SegmentationSegmentationUnsupervised Chinese Word Segmentation with BERT Oriented Probing and Transformation
Word Segmentation is a fundamental step for understanding Chinese language. Previous neural approaches for unsupervised Chinese Word Segmentation (CWS) only exploits shallow semantic information, which can miss important…
Chinese Word SegmentationSegmentation