Papers audio-visual event localization
“audio-visual event localization” 태그가 달린 논문 32편 · 필터 해제
Hierarchical Semantic-Constrained Heterogeneous Graph for Audio-Visual Event Localization
Open-vocabulary audio-visual event localization (OV-AVEL) jointly models audio-visual cues to recognize and temporally localize events, including categories unseen during training. Existing methods primarily learn joint …
audio-visual event localizationRA-SSU: Towards Fine-Grained Audio-Visual Learning with Region-Aware Sound Source Understanding
Audio-Visual Learning (AVL) is one fundamental task of multi-modality learning and embodied intelligence, displaying the vital role in scene understanding and interaction. However, previous researchers mostly focus on ex…
audio-visual event localizationSound Source LocalizationScene UnderstandingMoLT: Mixture of Layer-Wise Tokens for Efficient Audio-Visual Learning
In this paper, we propose Mixture of Layer-Wise Tokens (MoLT), a parameter- and memory-efficient adaptation framework for audio-visual learning. The key idea of MoLT is to replace conventional, computationally heavy sequ…
audio-visual event localizationAudio-visual Question AnsweringReal-Time Inference for Distributed Multimodal Systems under Communication Delay Uncertainty
Connected cyber-physical systems perform inference based on real-time inputs from multiple data streams. Uncertain communication delays across data streams challenge the temporal flow of the inference process. State-of-t…
audio-visual event localizationCLASP: Cross-modal Salient Anchor-based Semantic Propagation for Weakly-supervised Dense Audio-Visual Event Localization
The Dense Audio-Visual Event Localization (DAVEL) task aims to temporally localize events in untrimmed videos that occur simultaneously in both the audio and visual modalities. This paper explores DAVEL under a new and m…
audio-visual event localizationESG-Net: Event-Aware Semantic Guided Network for Dense Audio-Visual Event Localization
Dense audio-visual event localization (DAVE) aims to identify event categories and locate the temporal boundaries in untrimmed videos. Most studies only employ event-related semantic constraints on the final outputs, lac…
audio-visual event localizationAudio-visual Event Localization on Portrait Mode Short Videos
Audio-visual event localization (AVEL) plays a critical role in multimodal scene understanding. While existing datasets for AVEL predominantly comprise landscape-oriented long videos with clean and simple audio context, …
audio-visual event localizationScene UnderstandingAudio-Visual Semantic Graph Network for Audio-Visual Event Localization
Audio-visual event localization (AVEL) aims to identify both the category and temporal boundaries of events that are both audible and visible in unconstrained videos. However, the inherent semantic gap between hetero…
audio-visual event localizationcross-modal alignmentDense Audio-Visual Event Localization under Cross-Modal Consistency and Multi-Temporal Granularity Collaboration
In the field of audio-visual learning, most research tasks focus exclusively on short videos. This paper focuses on the more practical Dense Audio-Visual Event Localization (DAVEL) task, advancing audio-visual scene unde…
audio-visual event localizationaudio-visual learningScene UnderstandingPilot-guided Multimodal Semantic Communication for Audio-Visual Event Localization
Multimodal semantic communication, which integrates various data modalities such as text, images, and audio, significantly enhances communication efficiency and reliability. It has broad application prospects in fields s…
audio-visual event localizationAutonomous DrivingSemantic CommunicationTowards Open-Vocabulary Audio-Visual Event Localization
The Audio-Visual Event Localization (AVEL) task aims to temporally locate and classify video events that are both audible and visible. Most research in this field assumes a closed-set setting, which restricts these model…
audio-visual event localizationMultimodal Trustworthy Semantic Communication for Audio-Visual Event Localization
The exponential growth in wireless data traffic, driven by the proliferation of mobile devices and smart applications, poses significant challenges for modern communication systems. Ensuring the secure and reliable trans…
audio-visual event localizationSemantic CommunicationCACE-Net: Co-guidance Attention and Contrastive Enhancement for Effective Audio-Visual Event Localization
The audio-visual event localization task requires identifying concurrent visual and auditory events from unconstrained videos within a network model, locating them, and classifying their category. The efficient extractio…
audio-visual event localizationLabel-anticipated Event Disentanglement for Audio-Visual Video Parsing
Audio-Visual Video Parsing (AVVP) task aims to detect and temporally locate events within audio and visual modalities. Multiple events can overlap in the timeline, making identification challenging. While traditional met…
audio-visual event localizationDisentanglementSemantic SimilaritySemantic Textual SimilarityAdvancing Weakly-Supervised Audio-Visual Video Parsing via Segment-wise Pseudo Labeling
The Audio-Visual Video Parsing task aims to identify and temporally localize the events that occur in either or both the audio and visual streams of audible videos. It often performs in a weakly-supervised manner, where …
audio-visual event localizationDenoisingPseudo LabelUniAV: Unified Audio-Visual Perception for Multi-Task Video Event Localization
Video localization tasks aim to temporally locate specific instances in videos, including temporal action localization (TAL), sound event detection (SED) and audio-visual event localization (AVEL). Existing methods over-…
Action Localizationaudio-visual event localizationEvent DetectionSound Event Detection+1Temporal Label-Refinement for Weakly-Supervised Audio-Visual Event Localization
Audio-Visual Event Localization (AVEL) is the task of temporally localizing and classifying \emph{audio-visual events}, i.e., events simultaneously visible and audible in a video. In this paper, we solve AVEL in a weakly…
audio-visual event localizationModality-Independent Teachers Meet Weakly-Supervised Audio-Visual Event Parser
Audio-visual learning has been a major pillar of multi-modal machine learning, where the community mostly focused on its modality-aligned setting, i.e., the audio and visual modality are both assumed to signal the predic…
audio-visual event localizationaudio-visual learningDense-Localizing Audio-Visual Events in Untrimmed Videos: A Large-Scale Benchmark and Baseline
Existing audio-visual event localization (AVE) handles manually trimmed videos with only a single instance in each of them. However, this setting is unrealistic as natural videos often contain numerous audio-visual event…
audio-visual event localizationLeveraging the Video-level Semantic Consistency of Event for Audio-visual Event Localization
Audio-visual event (AVE) localization has attracted much attention in recent years. Most existing methods are often limited to independently encoding and classifying each video segment separated from the full video (whic…
audio-visual event localization