CDCPP:跨领域中文标点符号预测(CDCPP: Cross-Domain Chinese Punctuation Prediction)
标点符号对文本理解起很大作用。但目前,在中文文本特别是在社交媒体及问答领域文本中的标点符号使用存在非常多的错误或缺失的情况,这严重影响对其进行语义分析及机器翻译等各项自然语言处理的效果。当前对标点符号进行预测的相关研究多集中于英文对话的语音转写文本,缺少对社交媒体及问答领域文本进行标点预测的相关研究,也没有这些领域公开的数据集。本文首先提出跨领域中文标点符号预测任务,该任务是要利用标点符号基本规范正确的大规模新闻领域文本,建立标点符号预测模型,然后在标点符号标注不规范的社交媒体及问答领域,进行跨领域标点符号预测。随后构建了新闻、社交媒体及问答三个领域的相应数据集。最后还实现了一个基于BERT的标点符号预测基线模型,并在该数据集上进行了实验与分析。实验结果表明,直接利用新闻领域训练的模型,在社交媒体及问答领域上进行标点符号预测的性能均有所下降,在问答领域下降较小,在微博领域下降较大,超过20%,跨领域标点符号预测任务具有一定的挑战性。
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
A Small and Fast BERT for Chinese Medical Punctuation Restoration
In clinical dictation, utterances after automatic speech recognition (ASR) without explicit punctuation marks may lead to the misunderstanding of dictated reports. To give a precise and understandable clinical report wit…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Contrastive LearningPunctuation Restoration+2Predicting Punctuation in Ancient Chinese Texts: A Multi-Layered LSTM and Attention-Based Approach
It was only until the 20th century when the Chinese language began using punctuation. In fact, many ancient Chinese texts contain thousands of lines with no distinct punctuation marks or delimiters in sight. The lack of …
Transformer-Encoder-GRU (T-E-GRU) for Chinese Sentiment Analysis on Chinese Comment Text
Chinese sentiment analysis (CSA) has always been one of the challenges in natural language processing due to its complexity and uncertainty. Transformer has succeeded in capturing semantic features, but it uses position …
Chinese Sentiment AnalysisPositionSentenceSentence segmentation+1Average shortest-path length in word-adjacency networks: Chinese versus English
Complex networks provide powerful tools for analyzing and understanding the intricate structures present in various systems, including natural language. Here, we analyze topology of growing word-adjacency networks constr…