TNT: Text Normalization based Pre-training of Transformers for Content Moderation
In this work, we present a new language pre-training model TNT (Text Normalization based pre-training of Transformers) for content moderation. Inspired by the masking strategy and text normalization, TNT is developed to learn language representation by training transformers to reconstruct text from four operation types typically seen in text manipulation: substitution, transposition, deletion, and insertion. Furthermore, the normalization involves the prediction of both operation types and token labels, enabling TNT to learn from more challenging tasks than the standard task of masked word recovery. As a result, the experiments demonstrate that TNT outperforms strong baselines on the hate speech classification task. Additional text normalization experiments and case studies show that TNT is a new potential approach to misspelling correction.
Code (0)
등록된 구현이 없습니다.
Tasks
Text NormalizationSimilar Papers 제목 키워드 기반
Watermarking across Modalities for Content Tracing and Generative AI
Watermarking embeds information into digital content like images, audio, or text, imperceptible to humans but robustly detectable by specific algorithms. This technology has important applications in many challenges of t…
An Image is Worth a Thousand Toxic Words: A Metamorphic Testing Framework for Content Moderation Software
The exponential growth of social media platforms has brought about a revolution in communication and content dissemination in human society. Nevertheless, these platforms are being increasingly misused to spread toxic co…
The Potential of Vision-Language Models for Content Moderation of Children's Videos
Natural language supervision has been shown to be effective for zero-shot learning in many computer vision tasks, such as object detection and activity recognition. However, generating informative prompts can be challeng…
Activity Recognitionobject-detectionZero-Shot LearningRevealing Hidden Mechanisms of Cross-Country Content Moderation with Natural Language Processing
The ability of Natural Language Processing (NLP) methods to categorize text into multiple classes has motivated their use in online content moderation tasks, such as hate speech and fake news detection. However, there is…
Fake News DetectionMTTM: Metamorphic Testing for Textual Content Moderation Software
The exponential growth of social media platforms such as Twitter and Facebook has revolutionized textual communication and textual content publication in human society. However, they have been increasingly exploited to p…
Sentence