paper-with-me

홈 › Papers

Approaching Neural Chinese Word Segmentation as a Low-Resource Machine Translation Task

2020-08-12 · Pin-zhen Chen, Kenneth Heafield

Chinese word segmentation has entered the deep learning era which greatly reduces the hassle of feature engineering. Recently, some researchers attempted to treat it as character-level translation, which further simplified model designing, but there is a performance gap between the translation-based approach and other methods. This motivates our work, in which we apply the best practices from low-resource neural machine translation to supervised Chinese segmentation. We examine a series of techniques including regularization, data augmentation, objective weighting, transfer learning, and ensembling. Compared to previous works, our low-resource translation-based method maintains the effortless model design, yet achieves the same result as state of the art in the constrained evaluation without using additional data.

📄 PDF Abstract BibTeX arXiv:2008.05348

Code (1)

marian-nmt/marian 공식 구현

Tasks

Chinese Word SegmentationLow Resource Neural Machine TranslationLow-Resource Neural Machine TranslationMachine TranslationSegmentationTransfer Learning

Similar Papers 제목 키워드 기반

BERT 4EVER@EvaHan 2022: Ancient Chinese Word Segmentation and Part-of-Speech Tagging Based on Adversarial Learning and Continual Pre-training

2022-06-01 · LT4HALA (LREC) 2022 6 · Hailin Zhang, Ziyu Yang, Yingwen Fu, Ruoyao Ding

With the development of artificial intelligence (AI) and digital humanities, ancient Chinese resources and language technology have also developed and grown, which have become an increasingly important part to the study …

Chinese Word SegmentationCultural Vocal Bursts Intensity PredictionEnsemble LearningPart-Of-Speech Tagging+3

Transfer Deep Learning for Low-Resource Chinese Word Segmentation with a Novel Neural Network

2017-02-15 · Jingjing Xu, Xu sun

Recent studies have shown effectiveness in using neural networks for Chinese word segmentation. However, these models rely on large-scale data and are less effective for low-resource datasets because of insufficient trai…

Chinese Word SegmentationSegmentationTransfer Learning

Overview of the NLPCC 2015 Shared Task: Chinese Word Segmentation and POS Tagging for Micro-blog Texts

2015-05-28 · Xipeng Qiu, Peng Qian, Liusong Yin, Shiyu Wu 외

In this paper, we give an overview for the shared task at the 4th CCF Conference on Natural Language Processing \& Chinese Computing (NLPCC 2015): Chinese word segmentation and part-of-speech (POS) tagging for micro-blog…

Chinese Word SegmentationPart-Of-Speech TaggingPOSPOS Tagging+1

Improving Patent Translation using Bilingual Term Extraction and Re-tokenization for Chinese--Japanese

2016-12-01 · WS 2016 12 · Wei Yang, Yves Lepage

Unlike European languages, many Asian languages like Chinese and Japanese do not have typographic boundaries in written system. Word segmentation (tokenization) that break sentences down into individual words (tokens) is…

Chinese Word SegmentationMachine TranslationSegmentationTerm Extraction+1

Chinese Word Boundary Recovery through Character Alignment Projection

2026-05-27 · Lusha Wang, Yuchen Li, Su Yuan, Jungyeul Park arxiv

Chinese word segmentation is especially fragile in non-standard text, where language learner errors and other character-level divergences disrupt the word boundaries assumed by downstream annotation and evaluation. This …

Chinese Word Segmentation