Papers Audio-Visual Captioning
“Audio-Visual Captioning” 태그가 달린 논문 4편 · 필터 해제
LAVCap: LLM-based Audio-Visual Captioning using Optimal Transport
Automated audio captioning is a task that generates textual descriptions for audio content, and recent studies have explored using visual information to enhance captioning quality. However, current methods often fail to …
AudioCapsAudio captioningAudio-Visual CaptioningImage Captioning+3AVCap: Leveraging Audio-Visual Features as Text Tokens for Captioning
In recent years, advancements in representation learning and language models have propelled Automated Captioning (AC) to new heights, enabling the generation of human-level descriptions. Leveraging these advancements, we…
Audio-Visual CaptioningImage CaptioningRepresentation LearningVAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset
Vision and text have been fully explored in contemporary video-text foundational models, while other modalities such as audio and subtitles in videos have not received sufficient attention. In this paper, we resort to es…
Audio captioningAudio-Visual CaptioningAudio-visual Question AnsweringCross-Modal Retrieval+12VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset
In this paper, we propose a Vision-Audio-Language Omni-peRception pretraining model (VALOR) for multi-modal understanding and generation. Different from widely-studied vision-language pretraining models, VALOR jointly mo…
Audio captioningAudio-Video Question Answering (AVQA)Audio-Visual CaptioningAudio-visual Question Answering+16