Pre-training with Meta Learning for Chinese Word Segmentation
Recent researches show that pre-trained models (PTMs) are beneficial to Chinese Word Segmentation (CWS). However, PTMs used in previous works usually adopt language modeling as pre-training tasks, lacking task-specific prior segmentation knowledge and ignoring the discrepancy between pre-training tasks and downstream CWS tasks. In this paper, we propose a CWS-specific pre-trained model METASEG, which employs a unified architecture and incorporates meta learning algorithm into a multi-criteria pre-training task. Empirical results show that METASEG could utilize common prior segmentation knowledge from different existing criteria and alleviate the discrepancy between pre-trained models and downstream CWS tasks. Besides, METASEG can achieve new state-of-the-art performance on twelve widely-used CWS datasets and significantly improve model performance in low-resource settings.
Code (0)
등록된 구현이 없습니다.
Tasks
Chinese Word SegmentationLanguage ModelingLanguage ModellingMeta-LearningSegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MECT: Multi-Metadata Embedding based Cross-Transformer for Chinese Named Entity Recognition
Recently, word enhancement has become very popular for Chinese Named Entity Recognition (NER), reducing segmentation errors and increasing the semantic and boundary information of Chinese words. However, these methods te…
BenchmarkingChinese Named Entity Recognitionnamed-entity-recognitionNamed Entity Recognition+2BERT 4EVER@EvaHan 2022: Ancient Chinese Word Segmentation and Part-of-Speech Tagging Based on Adversarial Learning and Continual Pre-training
With the development of artificial intelligence (AI) and digital humanities, ancient Chinese resources and language technology have also developed and grown, which have become an increasingly important part to the study …
Chinese Word SegmentationCultural Vocal Bursts Intensity PredictionEnsemble LearningPart-Of-Speech Tagging+3Comparison of the Impact of Word Segmentation on Name Tagging for Chinese and Japanese
Word Segmentation is usually considered an essential step for many Chinese and Japanese Natural Language Processing tasks, such as name tagging. This paper presents several new observations and analysis on the impact of …
Chinese Word SegmentationSegmentationNeural Chinese Word Segmentation with Dictionary Knowledge
Chinese word segmentation (CWS) is an important task for Chinese NLP. Recently, many neural network based methods have been proposed for CWS. However, these methods require a large number of labeled sentences for model t…
Chinese Word SegmentationMulti-Task LearningSegmentationImproving Patent Translation using Bilingual Term Extraction and Re-tokenization for Chinese--Japanese
Unlike European languages, many Asian languages like Chinese and Japanese do not have typographic boundaries in written system. Word segmentation (tokenization) that break sentences down into individual words (tokens) is…
Chinese Word SegmentationMachine TranslationSegmentationTerm Extraction+1