Leveraging Recurrent Neural Networks for Multimodal Recognition of Social Norm Violation in Dialog
Social norms are shared rules that govern and facilitate social interaction. Violating such social norms via teasing and insults may serve to upend power imbalances or, on the contrary reinforce solidarity and rapport in conversation, rapport which is highly situated and context-dependent. In this work, we investigate the task of automatically identifying the phenomena of social norm violation in discourse. Towards this goal, we leverage the power of recurrent neural networks and multimodal information present in the interaction, and propose a predictive model to recognize social norm violation. Using long-term temporal and contextual information, our model achieves an F1 score of 0.705. Implications of our work regarding developing a social-aware agent are discussed.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
The MuSe 2024 Multimodal Sentiment Analysis Challenge: Social Perception and Humor Recognition
The Multimodal Sentiment Analysis Challenge (MuSe) 2024 addresses two contemporary multimodal affect and sentiment analysis problems: In the Social Perception Sub-Challenge (MuSe-Perception), participants will predict 16…
Humor DetectionMultimodal Sentiment AnalysisSentiment AnalysisTowards Robust and Realible Multimodal Misinformation Recognition with Incomplete Modality
Multimodal Misinformation Recognition has become an urgent task with the emergence of huge multimodal fake content on social media platforms. Previous studies mainly focus on complex feature extraction and fusion to lear…
Contrastive LearningA Heterogeneous Multimodal Graph Learning Framework for Recognizing User Emotions in Social Networks
The rapid expansion of social media platforms has provided unprecedented access to massive amounts of multimodal user-generated content. Comprehending user emotions can provide valuable insights for improving communicati…
Deep LearningEmotion RecognitionGraph LearningGraph Neural NetworkMultimodal Named Entity Recognition for Short Social Media Posts
We introduce a new task called Multimodal Named Entity Recognition (MNER) for noisy user-generated data such as tweets or Snapchat captions, which comprise short text with accompanying images. These social media posts of…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NERHear Me, See Me, Understand Me: Audio-Visual Autism Behavior Recognition
In this article, we introduce a novel problem of audio-visual autism behavior recognition, which includes social behavior recognition, an essential aspect previously omitted in AI-assisted autism screening research. We d…
Language ModellingLarge Language ModelMultimodal Large Language Model