Experimentation in Content Moderation using RWKV
This paper investigates the RWKV model's efficacy in content moderation through targeted experimentation. We introduce a novel dataset specifically designed for distillation into smaller models, enhancing content moderation practices. This comprehensive dataset encompasses images, videos, sounds, and text data that present societal challenges. Leveraging advanced Large Language Models (LLMs), we generated an extensive set of responses -- 558,958 for text and 83,625 for images -- to train and refine content moderation systems. Our core experimentation involved fine-tuning the RWKV model, capitalizing on its CPU-efficient architecture to address large-scale content moderation tasks. By highlighting the dataset's potential for knowledge distillation, this study not only demonstrates RWKV's capability in improving the accuracy and efficiency of content moderation systems but also paves the way for developing more compact, resource-efficient models in this domain. Datasets and models can be found in HuggingFace: https://huggingface.co/modrwkv
Code (0)
등록된 구현이 없습니다.
Tasks
CPUKnowledge DistillationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
RWKV-TS: Beyond Traditional Recurrent Neural Network for Time Series Tasks
Traditional Recurrent Neural Network (RNN) architectures, such as LSTM and GRU, have historically held prominence in time series tasks. However, they have recently seen a decline in their dominant position across various…
Computational EfficiencyTime SeriesTime Series ForecastingToken-Level Marginalization for Multi-Label LLM Classifiers
This paper addresses the critical challenge of deriving interpretable confidence scores from generative language models (LLMs) when applied to multi-label content safety classification. While models like LLaMA Guard are …
MG-RWKV: Multi-Grained Context-Aware RWKV for Temporal Forgery Localization
Driven by Artificial Intelligence-Generated Content (AIGC), the authenticity of audio-visual content is facing severe challenges. Temporal Forgery Localization (TFL) aims to precisely identify manipulated segments within…
Let Community Rules Be Reflected in Online Content Moderation
Content moderation is a widely used strategy to prevent the dissemination of irregular information on social media platforms. Despite extensive research on developing automated models to support decision-making in conten…
Decision MakingSLM-Mod: Small Language Models Surpass LLMs at Content Moderation
Large language models (LLMs) have shown promise in many natural language understanding tasks, including content moderation. However, these models can be expensive to query in real-time and do not allow for a community-sp…
Language ModelingLanguage ModellingNatural Language Understanding