Lightweight Models for Multimodal Sequential Data
Human language encompasses more than just text; it also conveys emotions through tone and gestures. We present a case study of three simple and efficient Transformer-based architectures for predicting sentiment and emotion in multimodal data. The Late Fusion model merges unimodal features to create a multimodal feature sequence, the Round Robin model iteratively combines bimodal features using cross-modal attention, and the Hybrid Fusion model combines trimodal and unimodal features together to form a final feature sequence for predicting sentiment. Our experiments show that our small models are effective and outperform the publicly released versions of much larger, state-of-the-art multimodal sentiment analysis systems.
Code (0)
등록된 구현이 없습니다.
Tasks
Multimodal Sentiment AnalysisSentiment AnalysisSimilar Papers 제목 키워드 기반
Open-World Distributed Robot Self-Localization with Transferable Visual Vocabulary and Both Absolute and Relative Features
Visual robot self-localization is a fundamental problem in visual robot navigation and has been studied across various problem settings, including monocular and sequential localization. However, many existing studies foc…
Graph Neural NetworkRobot NavigationAuRA: Internalizing Audio Understanding into LLMs as LoRA
Recent efforts to extend large language models (LLMs) to speech inputs typically rely on cascaded ASR-LLM pipelines, end-to-end speech-language models, or bridge/distillation-based adaptation. While these routes respecti…
CoRe-MoE: Compact Reusable MoE for Continual Multimodal Instruction Tuning
Continual multimodal instruction tuning requires multimodal large language models to acquire new task abilities sequentially while preserving previously learned knowledge. LoRA-MoE provides a promising solution by introd…
DocSLM: A Small Vision-Language Model for Long Multimodal Document Understanding
Large Vision-Language Models (LVLMs) have demonstrated strong multimodal reasoning capabilities on long and complex documents. However, their high memory footprint makes them impractical for deployment on resource-constr…
Multimodal ReasoningUnveiling the Potential of Multimodal Retrieval Augmented Generation with Planning
Multimodal Retrieval Augmented Generation (MRAG) systems, while promising for enhancing Multimodal Large Language Models (MLLMs), often rely on rigid, single-step retrieval methods. This limitation hinders their ability …
RetrievalRetrieval-augmented Generation