Exploring the Use of Abusive Generative AI Models on Civitai
The rise of generative AI is transforming the landscape of digital imagery, and exerting a significant influence on online creative communities. This has led to the emergence of AI-Generated Content (AIGC) social platforms, such as Civitai. These distinctive social platforms allow users to build and share their own generative AI models, thereby enhancing the potential for more diverse artistic expression. Designed in the vein of social networks, they also provide artists with the means to showcase their creations (generated from the models), engage in discussions, and obtain feedback, thus nurturing a sense of community. Yet, this openness also raises concerns about the abuse of such platforms, e.g., using models to disseminate deceptive deepfakes or infringe upon copyrights. To explore this, we conduct the first comprehensive empirical study of an AIGC social platform, focusing on its use for generating abusive content. As an exemplar, we construct a comprehensive dataset covering Civitai, the largest available AIGC social platform. Based on this dataset of 87K models and 2M images, we explore the characteristics of content and discuss strategies for moderation to better govern these platforms.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Exploring the Use of Lexicons to aid Deep Learning towards the Detection of Abusive Language
Detecting abusive language is a significant research topic, which has received a lot of attention recently. Our work focused on detecting personal attacks in online conversations. State-of-the-art research on this task h…
Abusive LanguageWord EmbeddingsDeepfakes on Demand: the rise of accessible non-consensual deepfake image generators
Advances in multimodal machine learning have made text-to-image (T2I) models increasingly accessible and popular. However, T2I models introduce risks such as the generation of non-consensual depictions of identifiable in…
Face Swappingparameter-efficient fine-tuningDetecting context abusiveness using hierarchical deep learning
Abusive text is a serious problem in social media and causes many issues among users as the number of users and the content volume increase. There are several attempts for detecting or preventing abusive text effectively…
Deep LearningUnraveling the Search Space of Abusive Language in Wikipedia with Dynamic Lexicon Acquisition
Many discussions on online platforms suffer from users offending others by using abusive terminology, threatening each other, or being sarcastic. Since an automatic detection of abusive language can support human moderat…
Abusive LanguageAbusive Language Detection in Heterogeneous Contexts: Dataset Collection and the Role of Supervised Attention
Abusive language is a massive problem in online social platforms. Existing abusive language detection techniques are particularly ill-suited to comments containing heterogeneous abusive language patterns, i.e., both abus…
Abusive LanguageMulti-Task Learning