Papers EgoSchema
“EgoSchema” 태그가 달린 논문 40편 · 필터 해제
Flash-VStream: Efficient Real-Time Understanding for Long Video Streams
Benefiting from the advances in large language models and cross-modal alignment, existing multimodal large language models have achieved prominent performance in image and short video understanding. However, the understa…
cross-modal alignmentEgoSchemaMMEMVBench+2EgoVLM: Policy Optimization for Egocentric Video Understanding
Emerging embodied AI applications, such as wearable cameras and autonomous agents, have underscored the need for robust reasoning from first person video streams. We introduce EgoVLM, a vision-language model specifically…
EgoSchemaQuestion Answeringreinforcement-learningReinforcement Learning+2Four Eyes Are Better Than Two: Harnessing the Collaborative Potential of Large Models via Differentiated Thinking and Complementary Ensembles
In this paper, we present the runner-up solution for the Ego4D EgoSchema Challenge at CVPR 2025 (Confirmed on May 20, 2025). Inspired by the success of large models, we evaluate and leverage leading accessible multimodal…
EgoSchemaFew-Shot LearningVideo UnderstandingRAVU: Retrieval Augmented Video Understanding with Compositional Reasoning over Graph
Comprehending long videos remains a significant challenge for Large Multi-modal Models (LMMs). Current LMMs struggle to process even minutes to hours videos due to their lack of explicit memory and retrieval mechanisms. …
EgoSchemaRetrievalVideo UnderstandingVideoMultiAgents: A Multi-Agent Framework for Video Question Answering
Video Question Answering (VQA) inherently relies on multimodal reasoning, integrating visual, temporal, and linguistic cues to achieve a deeper understanding of video content. However, many existing methods rely on feedi…
Caption GenerationEgoSchemaMultimodal ReasoningQuestion Answering+4Mobile-VideoGPT: Fast and Accurate Video Understanding Language Model
Video understanding models often struggle with high computational requirements, extensive parameter counts, and slow inference speed, making them inefficient for practical use. To tackle these challenges, we propose Mobi…
EgoSchemaLanguage ModelingLanguage ModellingMVBench+2LLaVAction: evaluating and training multi-modal large language models for action recognition
Understanding human behavior requires measuring behavioral actions. Due to its complexity, behavior is best mapped onto a rich, semantic structure such as language. The recent development of multi-modal large language mo…
Action RecognitionAction UnderstandingEgoSchemaMVBench+1Agentic Keyframe Search for Video Question Answering
Video question answering (VideoQA) enables machines to extract and comprehend key information from videos through natural language interaction, which is a critical step towards achieving intelligence. However, the demand…
EgoSchemaQuestion AnsweringVideo Question AnsweringVideo Understanding+2Keyframe-oriented Vision Token Pruning: Enhancing Efficiency of Large Vision Language Models on Long-Form Video Processing
Vision language models (VLMs) demonstrate strong capabilities in jointly processing visual and textual data. However, they often incur substantial computational overhead due to redundant visual information, particularly …
EgoSchemaFormVideo UnderstandingVLog: Video-Language Models by Generative Retrieval of Narration Vocabulary
Human daily activities can be concisely narrated as sequences of routine events (e.g., turning off an alarm) in video streams, forming an event vocabulary. Motivated by this, we introduce VLog, a novel video understandin…
EgoSchemaRetrievalVideo UnderstandingM-LLM Based Video Frame Selection for Efficient Video Understanding
Recent advances in Multi-Modal Large Language Models (M-LLMs) show promising results in video reasoning. Popular Multi-Modal Large Language Model (M-LLM) frameworks usually apply naive uniform sampling to reduce the numb…
EgoSchemaLanguage ModelingLanguage ModellingLarge Language Model+4MaxInfo: A Training-Free Key-Frame Selection Method Using Maximum Volume for Enhanced Video Understanding
Modern Video Large Language Models (VLLMs) often rely on uniform frame sampling for video understanding, but this approach frequently fails to capture critical information due to frame redundancy and variations in video …
DiversityEgoSchemaVideo UnderstandingUnderstanding Long Videos via LLM-Powered Entity Relation Graphs
The analysis of extended video content poses unique challenges in artificial intelligence, particularly when dealing with the complexity of tracking and understanding visual elements across time. Current methodologies th…
EgoSchemaLarge Language ModelObject TrackingRelation+1ENTER: Event Based Interpretable Reasoning for VideoQA
In this paper, we present ENTER, an interpretable Video Question Answering (VideoQA) system based on event graphs. Event graphs convert videos into graphical representations, where video events form the nodes and event-e…
Code GenerationEgoSchemaQuestion AnsweringVideo Question Answering+1LongViTU: Instruction Tuning for Long-Form Video Understanding
This paper introduces LongViTU, a large-scale (~121k QA pairs, ~900h videos), automatically generated dataset for long-form video understanding. We propose a systematic approach that organizes videos into a hierarchical …
EgoSchemaFormInstruction FollowingVideo UnderstandingBuilding a Mind Palace: Structuring Environment-Grounded Semantic Graphs for Effective Long Video Analysis with LLMs
Long-form video understanding with Large Vision Language Models is challenged by the need to analyze temporally dispersed yet spatially concentrated key moments within limited context windows. In this work, we introduce …
EgoSchemaObject TrackingVideo UnderstandingLyra: An Efficient and Speech-Centric Framework for Omni-Cognition
As Multi-modal Large Language Models (MLLMs) evolve, expanding beyond single-domain capabilities is essential to meet the demands for more versatile and efficient AI. However, previous omni-models have insufficiently exp…
EgoSchemaMMEMM-Vet+5Espresso: High Compression For Rich Extraction From Videos for Your Vision-Language Model
Recent advances in vision-language models (VLMs) have shown great promise in connecting images and text, but extending these models to long videos remains challenging due to the rapid growth in token counts. Models that …
EgoSchemaLanguage ModelingLanguage ModellingVideo UnderstandingVideoSAVi: Self-Aligned Video Language Models without Human Supervision
Recent advances in video-large language models (Video-LLMs) have led to significant progress in video understanding. Current preference optimization methods often rely on proprietary APIs or ground-truth captions to gene…
EgoSchemaMVBenchSpatial ReasoningVideo UnderstandingTimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning
Multimodal Large Language Models (MLLMs) have demonstrated impressive performance in short video understanding. However, understanding long-form videos still remains challenging for MLLMs. This paper proposes TimeSuite, …
EgoSchemaHallucinationHighlight DetectionMoment Retrieval+3