Improving Sequence Tagging for Vietnamese Text Using Transformer-based Neural Models
This paper describes our study on using mutilingual BERT embeddings and some new neural models for improving sequence tagging tasks for the Vietnamese language. We propose new model architectures and evaluate them extensively on two named entity recognition datasets of VLSP 2016 and VLSP 2018, and on two part-of-speech tagging datasets of VLSP 2010 and VLSP 2013. Our proposed models outperform existing methods and achieve new state-of-the-art results. In particular, we have pushed the accuracy of part-of-speech tagging to 95.40% on the VLSP 2010 corpus, to 96.77% on the VLSP 2013 corpus; and the F1 score of named entity recognition to 94.07% on the VLSP 2016 corpus, to 90.31% on the VLSP 2018 corpus. Our code and pre-trained models viBERT and vELECTRA are released as open source to facilitate adoption and further research.
Code (2)
Tasks
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)Part-Of-Speech TaggingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Neural sequence labeling for Vietnamese POS Tagging and NER
This paper presents a neural architecture for Vietnamese sequence labeling tasks including part-of-speech (POS) tagging and named entity recognition (NER). We applied the model described in \cite{lample-EtAl:2016:N16-1} …
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER+4An Encoder-Integrated PhoBERT with Graph Attention for Vietnamese Token-Level Classification
We propose a novel neural architecture named TextGraphFuseGAT, which integrates a pretrained transformer encoder (PhoBERT) with Graph Attention Networks for token-level classification tasks. The proposed model constructs…
ViSoBERT: A Pre-Trained Language Model for Vietnamese Social Media Text Processing
English and Chinese, known as resource-rich languages, have witnessed the strong development of transformer-based language models for natural language processing tasks. Although Vietnam has approximately 100M people spea…
Language ModelingLanguage ModellingVietnamese Hate Speech DetectionVietnamese Language Models+2From Word Segmentation to POS Tagging for Vietnamese
This paper presents an empirical comparison of two strategies for Vietnamese Part-of-Speech (POS) tagging from unsegmented text: (i) a pipeline strategy where we consider the output of a word segmenter as the input of a …
Part-Of-Speech TaggingPOSPOS TaggingTAGVieSum: How Robust Are Transformer-based Models on Vietnamese Summarization?
Text summarization is a challenging task within natural language processing that involves text generation from lengthy input sequences. While this task has been widely studied in English, there is very limited research o…
Abstractive Text SummarizationDecoderSelf-Supervised LearningText Generation+3