Vietnamese Word Segmentation with SVM: Ambiguity Reduction and Suffix Capture

2020-06-14 · Duc-Vu Nguyen, Dang Van Thin, Kiet Van Nguyen, Ngan Luu-Thuy Nguyen

In this paper, we approach Vietnamese word segmentation as a binary classification by using the Support Vector Machine classifier. We inherit features from prior works such as n-gram of syllables, n-gram of syllable types, and checking conjunction of adjacent syllables in the dictionary. We propose two novel ways to feature extraction, one to reduce the overlap ambiguity and the other to increase the ability to predict unknown words containing suffixes. Different from UETsegmenter and RDRsegmenter, two state-of-the-art Vietnamese word segmentation methods, we do not employ the longest matching algorithm as an initial processing step or any post-processing technique. According to experimental results on benchmark Vietnamese datasets, our proposed method obtained a better F1-score than the prior state-of-the-art methods UETsegmenter, and RDRsegmenter.

📄 PDF Abstract BibTeX arXiv:2006.07804

Code (1)

ngannlt/UITws-v1 공식 구현

Tasks

Binary Classification SegmentationVietnamese DatasetsVietnamese Word Segmentation

State-of-the-Art Vietnamese Word Segmentation

2019-06-18 · Song Nguyen Duc Cong, Quoc Hung Ngo, Rachsuda Jiamthapthaksin

Word segmentation is the first step of any tasks in Vietnamese language processing. This paper reviews stateof-the-art approaches and systems for word segmentation in Vietnamese. To have an overview of all stages from bu…

BIG-bench Machine Learning Segmentation Vietnamese Word Segmentation

Is word segmentation necessary for Vietnamese sentiment classification?

2023-01-01 · Duc-Vu Nguyen, Ngan Luu-Thuy Nguyen

To the best of our knowledge, this paper made the first attempt to answer whether word segmentation is necessary for Vietnamese sentiment classification. To do this, we presented five pre-trained monolingual S4- based la…

Classification Segmentation Sentiment Analysis Sentiment Classification

A hybrid approach to Vietnamese word segmentation

2016-12-29 · Tuan-Phong Nguyen, Anh-Cuong Le

Word segmentation is the very first task for Vietnamese language processing. Word-segmented text is the input of almost other NLP tasks. This task faces some challenges due to specific characteristics of the language. As…

Segmentation Sentence Vietnamese Word Segmentation

A neural joint model for Vietnamese word segmentation, POS tagging and dependency parsing

2018-12-30 · ALTA 2019 4 · Dat Quoc Nguyen

We propose the first multi-task learning model for joint Vietnamese word segmentation, part-of-speech (POS) tagging and dependency parsing. In particular, our model extends the BIST graph-based dependency parser (Kiperwa…

Dependency Parsing Multi-Task Learning Part-Of-Speech Tagging POS+3

Augmenting Part-of-speech Tagging with Syntactic Information for Vietnamese and Chinese

2021-02-24 · Duc-Vu Nguyen, Kiet Van Nguyen, Ngan Luu-Thuy Nguyen

Word segmentation and part-of-speech tagging are two critical preliminary steps for downstream tasks in Vietnamese natural language processing. In reality, people tend to consider also the phrase boundary when performing…

Part-Of-Speech Tagging Segmentation