Papers Audio-visual Question Answering
“Audio-visual Question Answering” 태그가 달린 논문 41편 · 필터 해제
LEAP: Learned Block-wise Evidence Retrieval for Long Audio-Video Perception
Hour-scale audio-visual question answering is constrained by a context dilemma: dense whole-recording encoding rapidly exhausts context limits, whereas uniform temporal compression severely dilutes fine-grained acoustic …
Audio-visual Question AnsweringReasoning for Social Audio-Visual Question Answering: Where Do We Stand?
Training Multimodal Large Language Models for audio-visual social understanding is a crucial step toward embodied social intelligence. Chain-of-thought (CoT) reasoning has become the dominant approach, with HumanOmniV2 a…
Audio-visual Question AnsweringQ-TriM: Question-Guided Tri-Modal Attention for Audio-Visual Question Answering
Audio-Visual Question Answering (AVQA) extends classical VQA by requiring joint reasoning over video and synchronized audio. However, many AVQA systems rely on deeply stacked layers of self- and cross attention across te…
Audio-visual Question AnsweringOmniVideo-100K: A Dataset for Audio-Visual Reasoning through Structured Scripts and Evidence Chains
Current automated pipelines for audio-visual Question Answering (QA) generally adopt a ``video-caption-QA'' paradigm. However, these methods typically segment videos into short clips and generate separate descriptions fo…
Audio-visual Question AnsweringVisual ReasoningSeparate First, Fuse Later: Mitigating Cross-Modal Interference in Audio-Visual LLMs Reasoning with Modality-Specific Chain-of-Thought
Audio and vision provide complementary evidence for audio-visual question answering, yet current audio-visual large language models may suffer from cross-modal interference: information from one modality misguides the in…
Audio-visual Question AnsweringReinforcement LearningVisual ReasoningRetrieving to Recover: Towards Incomplete Audio-Visual Question Answering via Semantic-consistent Purification
Recent Audio-Visual Question Answering (AVQA) methods have advanced significantly. However, most AVQA methods lack effective mechanisms for handling missing modalities, suffering from severe performance degradation in re…
Audio-visual Question AnsweringCross-Modal RetrievalKnowing When to Answer: Adaptive Confidence Refinement for Reliable Audio-Visual Question Answering
We present a formal problem formulation for \textit{Reliable} Audio-Visual Question Answering ($\mathcal{R}$-AVQA), where we prefer abstention over answering incorrectly. While recent AVQA models have high accuracy, thei…
Audio-visual Question AnsweringQuery-Guided Spatial-Temporal-Frequency Interaction for Music Audio-Visual Question Answering
Audio--Visual Question Answering (AVQA) is a challenging multimodal task that requires jointly reasoning over audio, visual, and textual information in a given video to answer natural language questions. Inspired by rece…
Audio-visual Question AnsweringMulti-Modal Scene Graph with Kolmogorov-Arnold Experts for Audio-Visual Question Answering
In this paper, we propose a novel Multi-Modal Scene Graph with Kolmogorov-Arnold Expert Network for Audio-Visual Question Answering (SHRIKE). The task aims to mimic human reasoning by extracting and fusing information fr…
Audio-visual Question AnsweringMoLT: Mixture of Layer-Wise Tokens for Efficient Audio-Visual Learning
In this paper, we propose Mixture of Layer-Wise Tokens (MoLT), a parameter- and memory-efficient adaptation framework for audio-visual learning. The key idea of MoLT is to replace conventional, computationally heavy sequ…
audio-visual event localizationAudio-visual Question AnsweringAccKV: Towards Efficient Audio-Video LLMs Inference via Adaptive-Focusing and Cross-Calibration KV Cache Optimization
Recent advancements in Audio-Video Large Language Models (AV-LLMs) have enhanced their capabilities in tasks like audio-visual question answering and multimodal dialog systems. Video and audio introduce an extended tempo…
Audio-visual Question AnsweringComputational EfficiencyWhen Eyes and Ears Disagree: Can MLLMs Discern Audio-Visual Confusion?
Can Multimodal Large Language Models (MLLMs) discern confused objects that are visually present but audio-absent? To study this, we introduce a new benchmark, AV-ConfuseBench, which simulates an ``Audio-Visual Confusion'…
Audio-visual Question AnsweringReinforcement LearningVisual ReasoningAV-Master: Dual-Path Comprehensive Perception Makes Better Audio-Visual Question Answering
Audio-Visual Question Answering (AVQA) requires models to effectively utilize both visual and auditory modalities to answer complex and diverse questions about audio-visual scenes. However, existing methods lack sufficie…
Audio-visual Question AnsweringSightSound-R1: Cross-Modal Reasoning Distillation from Vision to Audio Language Models
While large audio-language models (LALMs) have demonstrated state-of-the-art audio understanding, their reasoning capability in complex soundscapes still falls behind large vision-language models (LVLMs). Compared to the…
Audio-visual Question AnsweringLearning Sparsity for Effective and Efficient Music Performance Question Answering
Music performances, characterized by dense and continuous audio as well as seamless audio-visual integration, present unique challenges for multimodal scene understanding and reasoning. Recent Music Performance Audio-Vis…
Audio-visual Question AnsweringQuestion AnsweringScene UnderstandingSparse Learning+1Music's Multimodal Complexity in AVQA: Why We Need More than General Multimodal LLMs
While recent Multimodal Large Language Models exhibit impressive capabilities for general multimodal tasks, specialized domains like music necessitate tailored approaches. Music Audio-Visual Question Answering (Music AVQ…
Audio-visual Question AnsweringQuestion AnsweringVisual Question AnsweringFortisAVQA and MAVEN: a Benchmark Dataset and Debiasing Framework for Robust Multimodal Reasoning
Audio-Visual Question Answering (AVQA) is a challenging multimodal reasoning task requiring intelligent systems to answer natural language queries based on paired audio-video inputs accurately. However, existing AVQA app…
Audio-visual Question AnsweringAudio-Visual Question Answering (AVQA)Multimodal ReasoningNatural Language Queries+2PAVE: Patching and Adapting Video Large Language Models
Pre-trained video large language models (Video LLMs) exhibit remarkable reasoning capabilities, yet adapting these models to new tasks involving additional modalities or data types (e.g., audio or 3D information) remains…
Audio-visual Question AnsweringMulti-Task LearningQuestion AnsweringVideo Recognition+2Question-Aware Gaussian Experts for Audio-Visual Question Answering
Audio-Visual Question Answering (AVQA) requires not only question-based multimodal reasoning but also precise temporal grounding to capture subtle dynamics for accurate prediction. However, existing methods mainly use qu…
Audio-visual Question AnsweringAudio-Visual Question Answering (AVQA)AUDIO-VISUAL QUESTION ANSWERING (MUSIC-AVQA-v2.0)Mixture-of-Experts+3AVQACL: A Novel Benchmark for Audio-Visual Question Answering Continual Learning
In this paper, a novel benchmark for audio-visual question answering continual learning (AVQACL) is introduced, aiming to study fine-grained scene understanding and spatial-temporal reasoning in videos under a contin…
Audio-visual Question AnsweringContinual LearningKnowledge DistillationQuestion Answering+2