Self-supervised Video Object Segmentation with Distillation Learning of Deformable Attention
Video object segmentation is a fundamental research problem in computer vision. Recent techniques have often applied attention mechanism to object representation learning from video sequences. However, due to temporal changes in the video data, attention maps may not well align with the objects of interest across video frames, causing accumulated errors in long-term video processing. In addition, existing techniques have utilised complex architectures, requiring highly computational complexity and hence limiting the ability to integrate video object segmentation into low-powered devices. To address these issues, we propose a new method for self-supervised video object segmentation based on distillation learning of deformable attention. Specifically, we devise a lightweight architecture for video object segmentation that is effectively adapted to temporal changes. This is enabled by deformable attention mechanism, where the keys and values capturing the memory of a video sequence in the attention module have flexible locations updated across frames. The learnt object representations are thus adaptive to both the spatial and temporal dimensions. We train the proposed architecture in a self-supervised fashion through a new knowledge distillation paradigm where deformable attention maps are integrated into the distillation loss. We qualitatively and quantitatively evaluate our method and compare it with existing methods on benchmark datasets including DAVIS 2016/2017 and YouTube-VOS 2018/2019. Experimental results verify the superiority of our method via its achieved state-of-the-art performance and optimal memory usage.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge DistillationObjectRepresentation LearningSegmentationSemantic SegmentationUnsupervised Video Object SegmentationVideo Object SegmentationVideo Semantic SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Self-Distillation Embedded Supervised Affinity Attention Model for Few-Shot Segmentation
Few-shot segmentation focuses on the generalization of models to segment unseen object with limited annotated samples. However, existing approaches still face two main challenges. First, huge feature distinction between …
Few-Shot Semantic SegmentationSegmentationSemantic SegmentationAnomaly Detection in Video via Self-Supervised and Multi-Task Learning
Anomaly detection in video is a challenging computer vision problem. Due to the lack of anomalous events at training time, anomaly detection requires the design of learning methods without full supervision. In this paper…
Abnormal Event Detection In VideoAnomaly DetectionAnomaly Detection In Surveillance VideosEvent Detection+3PartDistillation: Learning Parts From Instance Segmentation
We present a scalable framework to learn part segmentation from object instance labels. State-of-the-art instance segmentation models contain a surprising amount of part information. However, much of this information…
Instance SegmentationObjectSegmentationSemantic Segmentation+1S2D: Sparse-To-Dense Keymask Distillation for Unsupervised Video Instance Segmentation
In recent years, the state-of-the-art in unsupervised video instance segmentation has heavily relied on synthetic video data, generated from object-centric image datasets such as ImageNet. However, video synthesis by art…
Unsupervised Instance SegmentationVideo Instance SegmentationMobileVOS: Real-Time Video Object Segmentation Contrastive Learning meets Knowledge Distillation
This paper tackles the problem of semi-supervised video object segmentation on resource-constrained devices, such as mobile phones. We formulate this problem as a distillation task, whereby we demonstrate that small spac…
Contrastive LearningKnowledge DistillationRepresentation LearningSemantic Segmentation+3