A New Clustering neural network for Chinese word segmentation
In this article I proposed a new model to achieve Chinese word segmentation(CWS),which may have the potentiality to apply in other domains in the future.It is a new thinking in CWS compared to previous works,to consider it as a clustering problem instead of a labeling problem.In this model,LSTM and self attention structures are used to collect context also sentence level features in every layer,and after several layers,a clustering model is applied to split characters into groups,which are the final segmentation results.I call this model CLNN.This algorithm can reach 98 percent of F score (without OOV words) and 85 percent to 95 percent F score (with OOV words) in training data sets.Error analyses shows that OOV words will greatly reduce performances,which needs a deeper research in the future.
Code (0)
등록된 구현이 없습니다.
Tasks
Chinese Word SegmentationClusteringSegmentationSentenceSimilar Papers 제목 키워드 기반
From Chinese Word Segmentation to Extraction of Constructions: Two Sides of the Same Algorithmic Coin
This paper presents the results of two experiments carried out within the framework of computational construction grammar. Starting from the constructionist point of view that there are just constructions in language, in…
Chinese Word SegmentationClusteringComparison of the Impact of Word Segmentation on Name Tagging for Chinese and Japanese
Word Segmentation is usually considered an essential step for many Chinese and Japanese Natural Language Processing tasks, such as name tagging. This paper presents several new observations and analysis on the impact of …
Chinese Word SegmentationSegmentationParsing Through Boundaries in Chinese Word Segmentation
Chinese word segmentation is a foundational task in natural language processing (NLP), with far-reaching effects on syntactic analysis. Unlike alphabetic languages like English, Chinese lacks explicit word boundaries, ma…
Chinese Word SegmentationSegmentationIncorporating Uncertain Segmentation Information into Chinese NER for Social Media Text
Chinese word segmentation is necessary to provide word-level information for Chinese named entity recognition (NER) systems. However, segmentation error propagation is a challenge for Chinese NER while processing colloqu…
Chinese Named Entity RecognitionChinese Word Segmentationnamed-entity-recognitionNamed Entity Recognition+5BERT 4EVER@EvaHan 2022: Ancient Chinese Word Segmentation and Part-of-Speech Tagging Based on Adversarial Learning and Continual Pre-training
With the development of artificial intelligence (AI) and digital humanities, ancient Chinese resources and language technology have also developed and grown, which have become an increasingly important part to the study …
Chinese Word SegmentationCultural Vocal Bursts Intensity PredictionEnsemble LearningPart-Of-Speech Tagging+3