3M: Multi-modal Multi-task Multi-teacher Learning for Game Event Detection
Esports has rapidly emerged as a global phenomenon with an ever-expanding audience via platforms, like YouTube. Due to the inherent complexity nature of the game, it is challenging for newcomers to comprehend what the event entails. The chaotic nature of online chat, the fast-paced speech of the game commentator, and the game-specific user interface further compound the difficulty for users in comprehending the gameplay. To overcome these challenges, it is crucial to integrate the Multi-Modal (MM) information from the platform and understand the event. The paper introduces a new MM multi-teacher-based game event detection framework, with the ultimate goal of constructing a comprehensive framework that enhances the comprehension of the ongoing game situation. While conventional MM models typically prioritise aligning MM data through concurrent training towards a unified objective, our framework leverages multiple teachers trained independently on different tasks to accomplish the Game Event Detection. The experiment clearly shows the effectiveness of the proposed MM multi-teacher framework.
Code (0)
등록된 구현이 없습니다.
Tasks
Event DetectionSimilar Papers 제목 키워드 기반
H-OPD: Confidence Aware Heterogeneous Multi-Teacher Multimodal On-policy Distillation
On-policy distillation (OPD) has recently emerged as an effective post-training paradigm by providing supervision on student-generated trajectories. However, existing OPD methods for multimodal reasoning usually rely on …
Multimodal ReasoningVisual GroundingAMMKD: Adaptive Multimodal Multi-teacher Distillation for Lightweight Vision-Language Models
The success of large-scale visual language pretraining (VLP) models has driven widespread adoption of image-text retrieval tasks. However, their deployment on mobile devices remains limited due to large model sizes and c…
Knowledge DistillationText RetrievalLearnable Cross-modal Knowledge Distillation for Multi-modal Learning with Missing Modality
The problem of missing modalities is both critical and non-trivial to be handled in multi-modal models. It is common for multi-modal tasks that certain modalities contribute more compared to other modalities, and if thos…
Knowledge DistillationMSD: Saliency-aware Knowledge Distillation for Multimodal Understanding
To reduce a model size but retain performance, we often rely on knowledge distillation (KD) which transfers knowledge from a large "teacher" model to a smaller "student" model. However, KD on multimodal datasets such as …
Knowledge DistillationMeta-LearningTelME: Teacher-leading Multimodal Fusion Network for Emotion Recognition in Conversation
Emotion Recognition in Conversation (ERC) plays a crucial role in enabling dialogue systems to effectively respond to user requests. The emotions in a conversation can be identified by the representations from various mo…
Emotion RecognitionEmotion Recognition in ConversationKnowledge DistillationLanguage Modeling+1