Unsupervised Chinese Word Segmentation with BERT Oriented Probing and Transformation
Word Segmentation is a fundamental step for understanding Chinese language. Previous neural approaches for unsupervised Chinese Word Segmentation (CWS) only exploits shallow semantic information, which can miss important context. Large scale Pre-trained language models (PLM) have achieved great success in many areas because of its ability to capture the deep contextual semantic relation. In this paper, we propose to take advantage of the deep semantic information embedded in PLM (e.g., BERT) with a self-training manner, which iteratively probes and transforms the semantic information in PLM into explicit word segmentation ability. Extensive experiment results show that our proposed approach achieves state-of-the-art F1 score on two CWS benchmark datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Chinese Word SegmentationSegmentationSimilar Papers 제목 키워드 기반
Unsupervised Chinese Word Segmentation with BERT Oriented Probing and Transformation
Word Segmentation is a fundamental step for understanding Chinese language. Previous neural approaches for unsupervised Chinese Word Segmentation (CWS) only exploits shallow semantic information, which can miss important…
Chinese Word SegmentationSegmentationUnsupervised Boundary-Aware Language Model Pretraining for Chinese Sequence Labeling
Boundary information is critical for various Chinese language processing tasks, such as word segmentation, part-of-speech tagging, and named entity recognition. Previous studies usually resorted to the use of a high-qual…
Chinese Named Entity RecognitionChinese Word SegmentationLanguage ModelingLanguage Modelling+3Character, Word, or Both? Revisiting the Segmentation Granularity for Chinese Pre-trained Language Models
Pretrained language models (PLMs) have shown marvelous improvements across various NLP tasks. Most Chinese PLMs simply treat an input text as a sequence of characters, and completely ignore word information. Although Who…
A Graph-based Model for Joint Chinese Word Segmentation and Dependency Parsing
Chinese word segmentation and dependency parsing are two fundamental tasks for Chinese natural language processing. The dependency parsing is defined on word-level. Therefore word segmentation is the precondition of depe…
Chinese Word SegmentationDependency ParsingFeature EngineeringLanguage Modeling+2Unified Multi-Criteria Chinese Word Segmentation with BERT
Multi-Criteria Chinese Word Segmentation (MCCWS) aims at finding word boundaries in a Chinese sentence composed of continuous characters while multiple segmentation criteria exist. The unified framework has been widely u…
Chinese Word SegmentationLanguage ModelingLanguage ModellingMulti-Task Learning+2