Papers Video Understanding
“Video Understanding” 태그가 달린 논문 1,149편 · 필터 해제
InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video Understanding
Modern multimodal large language models (MLLMs) can reason over hour-long video, yet their key-value (KV) cache grows linearly with time--quickly exceeding the fixed memory of phones, AR glasses, and edge robots. Prior c…
GPUStreaming video understandingTARVideo Understandingvideo-SALMONN 2: Captioning-Enhanced Audio-Visual Large Language Models
Videos contain a wealth of information, and generating detailed and accurate descriptions in natural language is a key aspect of video understanding. In this paper, we present video-SALMONN 2, an advanced audio-visual la…
Audio captioningLarge Language ModelQuestion AnsweringVideo Captioning+2EVA02-AT: Egocentric Video-Language Understanding with Spatial-Temporal Rotary Positional Embeddings and Symmetric Optimization
Egocentric video-language understanding demands both high efficiency and accurate spatial-temporal modeling. Existing approaches face three key challenges: 1) Excessive pre-training cost arising from multi-stage pre-trai…
Multi-Instance RetrievalRetrievalVideo UnderstandingMambaMia: A State-Space-Model-Based Compression for Efficient Video Understanding in Large Multimodal Models
We propose an efficient framework to compress multiple video-frame features before feeding them into large multimodal models, thereby mitigating the severe token explosion arising from long or dense videos. Our design le…
Video UnderstandingAdaVideoRAG: Omni-Contextual Adaptive Retrieval-Augmented Efficient Long Video Understanding
Multimodal Large Language Models (MLLMs) struggle with long videos due to fixed context windows and weak long-term dependency modeling. Existing Retrieval-Augmented Generation (RAG) methods for videos use static retrieva…
Optical Character Recognition (OCR)RAGRetrievalRetrieval-augmented Generation+1M^3-VOS: Multi-Phase, Multi-Transition, and Multi-Scenery Video Object Segmentation
Intelligent robots need to interact with diverse objects across various environments. The appearance and state of objects frequently undergo complex transformations depending on the object properties, e.g., phase transit…
ObjectSemantic SegmentationVideo Object SegmentationVideo Semantic Segmentation+2Self-supervised Learning of Echocardiographic Video Representations via Online Cluster Distillation
Self-supervised learning (SSL) has achieved major advances in natural images and video understanding, but challenges remain in domains like echocardiography (heart ultrasound) due to subtle anatomical structures, complex…
Anomaly DetectionClusteringRepresentation LearningSelf-Supervised Learning+2VideoDeepResearch: Long Video Understanding With Agentic Tool Using
Long video understanding (LVU) presents a significant challenge for current multi-modal large language models (MLLMs) due to the task's inherent complexity and context window constraint. It is widely assumed that address…
MMEVideo MMEVideo UnderstandingHopaDIFF: Holistic-Partial Aware Fourier Conditioned Diffusion for Referring Human Action Segmentation in Multi-Person Scenarios
Action segmentation is a core challenge in high-level video understanding, aiming to partition untrimmed videos into segments and assign each a label from a predefined action set. Existing methods primarily address singl…
Action RecognitionAction SegmentationBenchmarkingSegmentation+1MLVTG: Mamba-Based Feature Alignment and LLM-Driven Purification for Multi-Modal Video Temporal Grounding
Video Temporal Grounding (VTG), which aims to localize video clips corresponding to natural language queries, is a fundamental yet challenging task in video understanding. Existing Transformer-based methods often suffer …
Language ModelingLanguage ModellingLarge Language ModelMamba+2VersaVid-R1: A Versatile Video Understanding and Reasoning Model from Question Answering to Captioning Tasks
Recent advancements in multimodal large language models have successfully extended the Reason-Then-Respond paradigm to image-based reasoning, yet video-based reasoning remains an underdeveloped frontier, primarily due to…
Multiple-choiceOpen-Ended Question AnsweringQuestion AnsweringVideo Captioning+1SceneRAG: Scene-level Retrieval-Augmented Generation for Video Understanding
Despite recent advances in retrieval-augmented generation (RAG) for video understanding, effectively understanding long-form video content remains underexplored due to the vast scale and high complexity of video data. Cu…
RAGRetrievalRetrieval-augmented GenerationVideo UnderstandingCyberV: Cybernetics for Test-time Scaling in Video Understanding
Current Multimodal Large Language Models (MLLMs) may struggle with understanding long or complex videos due to computational demands at test time, lack of robustness, and limited accuracy, primarily stemming from their f…
Video UnderstandingSurgBench: A Unified Large-Scale Benchmark for Surgical Video Analysis
Surgical video understanding is pivotal for enabling automated intraoperative decision-making, skill assessment, and postoperative quality improvement. However, progress in developing surgical video foundation models (FM…
Action ClassificationBenchmarkingDecision MakingDomain Generalization+2Super Encoding Network: Recursive Association of Multi-Modal Encoders for Video Understanding
Video understanding has been considered as one critical step towards world modeling, which is an important long-term problem in AI research. Recently, multi-modal foundation models have shown such potential via large-sca…
Contrastive LearningVideo EditingVideo UnderstandingBridging Perspectives: A Survey on Cross-view Collaborative Intelligence with Egocentric-Exocentric Vision
Perceiving the world from both egocentric (first-person) and exocentric (third-person) perspectives is fundamental to human cognition, enabling rich and complementary understanding of dynamic environments. In recent year…
Video UnderstandingBridging Audio and Vision: Zero-Shot Audiovisual Segmentation by Connecting Pretrained Models
Audiovisual segmentation (AVS) aims to identify visual regions corresponding to sound sources, playing a vital role in video understanding, surveillance, and human-computer interaction. Traditional AVS methods depend on …
SegmentationVideo UnderstandingAV-Reasoner: Improving and Benchmarking Clue-Grounded Audio-Visual Counting for MLLMs
Despite progress in video understanding, current MLLMs struggle with counting tasks. Existing benchmarks are limited by short videos, close-set queries, lack of clue annotations, and weak multimodal coverage. In this pap…
BenchmarkingVideo UnderstandingDualX-VSR: Dual Axial Spatial$\times$Temporal Transformer for Real-World Video Super-Resolution without Motion Compensation
Transformer-based models like ViViT and TimeSformer have advanced video understanding by effectively modeling spatiotemporal dependencies. Recent video generation models, such as Sora and Vidu, further highlight the powe…
Motion CompensationOptical Flow EstimationSuper-ResolutionVideo Generation+2APVR: Hour-Level Long Video Understanding with Adaptive Pivot Visual Information Retrieval
Current video-based multimodal large language models struggle with hour-level video understanding due to computational constraints and inefficient information extraction from extensive temporal sequences. We propose APVR…
Information RetrievalRetrievalTemporal SequencesVideo Understanding