A Weakly Supervised Data Labeling Framework for Machine Lexical Normalization in Vietnamese Social Media
This study introduces an innovative automatic labeling framework to address the challenges of lexical normalization in social media texts for low-resource languages like Vietnamese. Social media data is rich and diverse, but the evolving and varied language used in these contexts makes manual labeling labor-intensive and expensive. To tackle these issues, we propose a framework that integrates semi-supervised learning with weak supervision techniques. This approach enhances the quality of training dataset and expands its size while minimizing manual labeling efforts. Our framework automatically labels raw data, converting non-standard vocabulary into standardized forms, thereby improving the accuracy and consistency of the training data. Experimental results demonstrate the effectiveness of our weak supervision framework in normalizing Vietnamese text, especially when utilizing Pre-trained Language Models. The proposed framework achieves an impressive F1-score of 82.72% and maintains vocabulary integrity with an accuracy of up to 99.22%. Additionally, it effectively handles undiacritized text under various conditions. This framework significantly enhances natural language normalization quality and improves the accuracy of various NLP tasks, leading to an average accuracy increase of 1-3%.
Code (0)
등록된 구현이 없습니다.
Tasks
Lexical NormalizationSimilar Papers 제목 키워드 기반
Uncertainty Based Active Learning Strategy for Interactive Weakly Supervised Learning through Data Programming
Easy to build and reliable machine learning models are what all data analysts want. Although machine learning is advancing daily, the labeling cost for supervised learning and the black-box nature of machine learning are…
Active LearningBIG-bench Machine Learningtext-classificationText Classification+1Reducing Label Dependency in Human Activity Recognition with Wearables: From Supervised Learning to Novel Weakly Self-Supervised Approaches
Human activity recognition (HAR) using wearable sensors has advanced through various machine learning paradigms, each with inherent trade-offs between performance and labeling requirements. While fully supervised techniq…
Human Activity RecognitionSelf-Supervised LearningMulti-Task LearningLearning from Reduced Labels for Long-Tailed Data
Long-tailed data is prevalent in real-world classification tasks and heavily relies on supervised information, which makes the annotation process exceptionally labor-intensive and time-consuming. Unfortunately, despite b…
Weakly-supervised LearningConstrained Labeling for Weakly Supervised Learning
Curation of large fully supervised datasets has become one of the major roadblocks for machine learning. Weak supervision provides an alternative to supervised learning by training with cheap, noisy, and possibly correla…
image-classificationImage ClassificationWeakly-supervised LearningBoxShrink: From Bounding Boxes to Segmentation Masks
One of the core challenges facing the medical image computing community is fast and efficient data sample labeling. Obtaining fine-grained labels for segmentation is particularly demanding since it is expensive, time-con…
Segmentation