Malicious or Benign? Towards Effective Content Moderation for Children's Videos
Online video platforms receive hundreds of hours of uploads every minute, making manual content moderation impossible. Unfortunately, the most vulnerable consumers of malicious video content are children from ages 1-5 whose attention is easily captured by bursts of color and sound. Scammers attempting to monetize their content may craft malicious children's videos that are superficially similar to educational videos, but include scary and disgusting characters, violent motions, loud music, and disturbing noises. Prominent video hosting platforms like YouTube have taken measures to mitigate malicious content on their platform, but these videos often go undetected by current content moderation tools that are focused on removing pornographic or copyrighted content. This paper introduces our toolkit Malicious or Benign for promoting research on automated content moderation of children's videos. We present 1) a customizable annotation tool for videos, 2) a new dataset with difficult to detect test cases of malicious content and 3) a benchmark suite of state-of-the-art video classification models.
Code (2)
Tasks
Video ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Enhanced Multimodal Content Moderation of Children's Videos using Audiovisual Fusion
Due to the rise in video content creation targeted towards children, there is a need for robust content moderation schemes for video hosting platforms. A video that is visually benign may include audio content that is in…
Prompt LearningRobust classificationThe Potential of Vision-Language Models for Content Moderation of Children's Videos
Natural language supervision has been shown to be effective for zero-shot learning in many computer vision tasks, such as object detection and activity recognition. However, generating informative prompts can be challeng…
Activity Recognitionobject-detectionZero-Shot LearningMoltbook Moderation: Uncovering Hidden Intent Through Multi-Turn Dialogue
The emergence of multi-agent systems introduces novel moderation challenges that extend beyond content filtering. Agents with malicious intent may contribute harmful content that appears benign to evade content-based mod…
An Image is Worth a Thousand Toxic Words: A Metamorphic Testing Framework for Content Moderation Software
The exponential growth of social media platforms has brought about a revolution in communication and content dissemination in human society. Nevertheless, these platforms are being increasingly misused to spread toxic co…
MTTM: Metamorphic Testing for Textual Content Moderation Software
The exponential growth of social media platforms such as Twitter and Facebook has revolutionized textual communication and textual content publication in human society. However, they have been increasingly exploited to p…
Sentence