Papers Temporal Localization
“Temporal Localization” 태그가 달린 논문 153편 · 필터 해제
Transformer-based Fusion of 2D-pose and Spatio-temporal Embeddings for Distracted Driver Action Recognition
Classification and localization of driving actions over time is important for advanced driver-assistance systems and naturalistic driving studies. Temporal localization is challenging because it requires robustness, reli…
2D Human Pose EstimationAction RecognitionPose EstimationTemporal Action Localization+1OLViT: Multi-Modal State Tracking via Attention-Based Embeddings for Video-Grounded Dialog
We present the Object Language Video Transformer (OLViT) - a novel model for video dialog operating over a multi-modal attention-based dialog state tracker. Existing video dialog models struggle with questions requiring …
ObjectObject TrackingResponse GenerationTemporal LocalizationSemi-supervised Active Learning for Video Action Detection
In this work, we focus on label efficient learning for video action detection. We develop a novel semi-supervised active learning approach which utilizes both labeled as well as unlabeled data along with informative samp…
Action DetectionActive LearningPseudo LabelSemantic Segmentation+5Deep-Learning-Assisted Analysis of Cataract Surgery Videos
Following the technological advancements in medicine, the operation rooms are evolving into intelligent environments. The context-aware systems (CAS) can comprehensively interpret the surgical state, enable real-time war…
Decision MakingDeep LearningRepresentation LearningSemantic Segmentation+1TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding
This work proposes TimeChat, a time-sensitive multimodal large language model specifically designed for long video understanding. Our model incorporates two key architectural contributions: (1) a timestamp-aware frame en…
Dense CaptioningHighlight DetectionInstruction FollowingLanguage Modeling+8Survey of Action Recognition, Spotting and Spatio-Temporal Localization in Soccer -- Current Trends and Research Perspectives
Action scene understanding in soccer is a challenging task due to the complex and dynamic nature of the game, as well as the interactions between players. This article provides a comprehensive overview of this task divid…
Action LocalizationAction RecognitionScene UnderstandingSpatio-Temporal Action Localization+2Cross-Video Contextual Knowledge Exploration and Exploitation for Ambiguity Reduction in Weakly Supervised Temporal Action Localization
Weakly supervised temporal action localization (WSTAL) aims to localize actions in untrimmed videos using video-level labels. Despite recent advances, existing approaches mainly follow a localization-by-classification pi…
Action LocalizationContrastive LearningSelf-LearningTemporal Action Localization+2UnLoc: A Unified Framework for Video Localization Tasks
While large-scale image-text pretrained models such as CLIP have been used for multiple video-level tasks on trimmed videos, their use for temporal localization in untrimmed videos is still a relatively unexplored task. …
Action SegmentationMoment RetrievalNatural Language Moment RetrievalRetrieval+3VideoGLUE: Video General Understanding Evaluation of Foundation Models
We evaluate the video understanding capabilities of existing foundation models (FMs) using a carefully designed experiment protocol consisting of three hallmark tasks (action recognition,temporal localization, and spatio…
Action RecognitionTemporal LocalizationVideo UnderstandingDense Video Object Captioning from Disjoint Supervision
We propose a new task and model for dense video object captioning -- detecting, tracking and captioning trajectories of objects in a video. This task unifies spatial and temporal localization in video, whilst also requir…
ObjectSentenceTemporal LocalizationVideo GroundingSingle-Stage Visual Query Localization in Egocentric Videos
Visual Query Localization on long-form egocentric videos requires spatio-temporal search and localization of visually specified objects and is vital to build episodic memory systems. Prior work develops complex multi-sta…
object-detectionObject DetectionTemporal LocalizationVideo RelationshipSelf-Chained Image-Language Model for Video Localization and Question Answering
Recent studies have shown promising results on utilizing large pre-trained image-language models for video question answering. While these image-language models can efficiently bootstrap the representation learning of vi…
Language ModelingLanguage ModellingQuestion AnsweringRepresentation Learning+3Autonomous Stabilization of Retinal Videos for Streamlining Assessment of Spontaneous Venous Pulsations
Spontaneous retinal Venous Pulsations (SVP) are rhythmic changes in the caliber of the central retinal vein and are observed in the optic disc region (ODR) of the retina. Its absence is a critical indicator of various oc…
Template MatchingTemporal LocalizationVideo StabilizationStructured Video-Language Modeling with Temporal Grouping and Spatial Grounding
Existing video-language pre-training methods primarily focus on instance-level alignment between video clips and captions via global contrastive learning but neglect rich fine-grained local information in both videos and…
Action LocalizationAction RecognitionContrastive LearningLanguage Modeling+7VADER: Video Alignment Differencing and Retrieval
We propose VADER, a spatio-temporal matching, alignment, and change summarization method to help fight misinformation spread via manipulated videos. VADER matches and coarsely aligns partial video fragments to candidate …
MisinformationRetrievalTemporal LocalizationVideo AlignmentUnsupervised classification to improve the quality of a bird song recording dataset
Open audio databases such as Xeno-Canto are widely used to build datasets to explore bird song repertoire or to train models for automatic bird sound classification by deep learning algorithms. However, such databases su…
Sound ClassificationTemporal LocalizationMulti-Task Learning of Object State Changes from Uncurated Videos
We aim to learn to temporally localize object state changes and the corresponding state-modifying actions by observing people interacting with objects in long uncurated web videos. We introduce three principal contributi…
Multi-Task LearningObjectSelf-Supervised LearningTemporal LocalizationMasked Autoencoders for Egocentric Video Understanding @ Ego4D Challenge 2022
In this report, we present our approach and empirical results of applying masked autoencoders in two egocentric video understanding tasks, namely, Object State Change Classification and PNR Temporal Localization, of Ego4…
Object State Change ClassificationTemporal LocalizationVideo UnderstandingExploring State Change Capture of Heterogeneous Backbones @ Ego4D Hands and Objects Challenge 2022
Capturing the state changes of interacting objects is a key technology for understanding human-object interactions. This technical report describes our method using heterogeneous backbones for the Ego4D Object State Chan…
Human-Object Interaction DetectionObjectObject State Change ClassificationTemporal Localization+1Optimizing Temporal Resolution Of Convolutional Recurrent Neural Networks For Sound Event Detection
In this technical report, the systems we submitted for subtask 4 of the DCASE 2021 challenge, regarding sound event detection, are described in detail. These models are closely related to the baseline provided for this p…
Event DetectionSound Event DetectionTemporal Localization