Beyond still images: Temporal features and input variance resilience
Traditionally, vision models have predominantly relied on spatial features extracted from static images, deviating from the continuous stream of spatiotemporal features processed by the brain in natural vision. While numerous video-understanding models have emerged, incorporating videos into image-understanding models with spatiotemporal features has been limited. Drawing inspiration from natural vision, which exhibits remarkable resilience to input changes, our research focuses on the development of a brain-inspired model for vision understanding trained with videos. Our findings demonstrate that models that train on videos instead of still images and include temporal features become more resilient to various alternations on input media.
Code (0)
등록된 구현이 없습니다.
Tasks
Video UnderstandingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Learning to Reason: Temporal Saliency Distillation for Interpretable Knowledge Transfer
Knowledge distillation has proven effective for model compression by transferring knowledge from a larger network called the teacher to a smaller network called the student. Current knowledge distillation in time series …
Knowledge DistillationTime Series AnalysisModel CompressionTemporal Hallucinating for Action Recognition With Few Still Images
Action recognition in still images has been recently promoted by deep learning. However, the success of these deep models heavily depends on huge amount of training images for various action categories, which may not be …
Action RecognitionAction Recognition In Still ImagesDomain AdaptationTemporal Action LocalizationTASeg: Temporal Aggregation Network for LiDAR Semantic Segmentation
Training deep models for LiDAR semantic segmentation is challenging due to the inherent sparsity of point clouds. Utilizing temporal data is a natural remedy against the sparsity problem as it makes the input signal dens…
LIDAR Semantic SegmentationSemantic SegmentationVideo Saliency Detection by 3D Convolutional Neural Networks
Different from salient object detection methods for still images, a key challenging for video saliency detection is how to extract and combine spatial and temporal features. In this paper, we present a novel and effectiv…
Objectobject-detectionObject DetectionRGB Salient Object Detection+5SwiftVLA: Unlocking Spatiotemporal Dynamics for Lightweight VLA Models at Minimal Overhead
Vision-Language-Action (VLA) models built on pretrained Vision-Language Models (VLMs) show strong potential but are limited in practicality due to their large parameter counts. To mitigate this issue, using a lightweight…