SED:Lightweight Saliency prediction for Event-based data via Distillation
Event-based saliency prediction has gained attention recently, as combining event cameras with saliency estimation can act as an upstream stage that naturally improves the efficiency of downstream eventbased perception at the edge. However, current approaches are either neuromorphic, underperforming on event-based saliency benchmarks, or too heavy for resource-constrained edge applications due to their reliance on transformers or 3D convolutions. Drawing inspiration from efficient convolutional modules, SED and aiming to exploit the temporal information in event data, we propose a lightweight network, trained through knowledge distillation, built on a Depthwise Spatio-Temporal Block (DSTconv) -- a factorization of the 3D depthwise separable convolution. Relative to its teacher, our model reduces the model size from 180 MB to 0.32 MB (562x) and the parameter count from 45M to 81k (554x), while matching or outperforming it on the N-DHF1K and N-UCF Sports datasets. Moreover, it generalizes strongly beyond its training distribution, transferring from synthetic to real event data where a model trained from scratch fails.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge DistillationSaliency PredictionSimilar Papers 제목 키워드 기반
Programmable Silicon Retina on Pixel Processor Array
Standard dynamic vision sensors approximate retinal processing by detecting temporal contrast changes, offering high speed and high dynamic range. In this work, we explore whether incorporating additional biologically in…
Saliency PredictionExploring deep learning for Event-Based Saliency Prediction with a Transformer-based model
Saliency prediction has been extensively studied in RGB images and videos as a computational model of human visual attention. In contrast, predicting saliency from event-based data remains largely unexplored, despite the…
Saliency PredictionEvent-based visionTinyHD: Efficient Video Saliency Prediction with Heterogeneous Decoders using Hierarchical Maps Distillation
Video saliency prediction has recently attracted attention of the research community, as it is an upstream task for several practical applications. However, current solutions are particularly computationally demanding, e…
Knowledge DistillationPredictionSaliency PredictionVideo Saliency PredictionSalNAS: Efficient Saliency-prediction Neural Architecture Search with self-knowledge distillation
Recent advancements in deep convolutional neural networks have significantly improved the performance of saliency prediction. However, the manual configuration of the neural network architectures requires domain knowledg…
DecoderKnowledge DistillationNeural Architecture SearchPrediction+2Beyond Background Bias: Saliency-Driven Prototype Alignment for Dataset Distillation
Dataset distillation aims to synthesize compact datasets that can approximate the performance of full-data training while significantly reducing computational and storage costs. However, diffusion-based distillation meth…