Memory Aggregation Networks for Efficient Interactive Video Object Segmentation
Interactive video object segmentation (iVOS) aims at efficiently harvesting high-quality segmentation masks of the target object in a video with user interactions. Most previous state-of-the-arts tackle the iVOS with two independent networks for conducting user interaction and temporal propagation, respectively, leading to inefficiencies during the inference stage. In this work, we propose a unified framework, named Memory Aggregation Networks (MA-Net), to address the challenging iVOS in a more efficient way. Our MA-Net integrates the interaction and the propagation operations into a single network, which significantly promotes the efficiency of iVOS in the scheme of multi-round interactions. More importantly, we propose a simple yet effective memory aggregation mechanism to record the informative knowledge from the previous interaction rounds, improving the robustness in discovering challenging objects of interest greatly. We conduct extensive experiments on the validation set of DAVIS Challenge 2018 benchmark. In particular, our MA-Net achieves the J@60 score of 76.1% without any bells and whistles, outperforming the state-of-the-arts with more than 2.7%.
Code (0)
등록된 구현이 없습니다.
Tasks
Interactive Video Object SegmentationObjectSemantic SegmentationVideo Object SegmentationVideo Semantic SegmentationSimilar Papers 제목 키워드 기반
MoSAM: Motion-Guided Segment Anything Model with Spatial-Temporal Memory Selection
The recent Segment Anything Model 2 (SAM2) has demonstrated exceptional capabilities in interactive object segmentation for both images and videos. However, as a foundational model on interactive segmentation, SAM2 perfo…
Instance SegmentationInteractive SegmentationObjectObject Tracking+5Alignment Before Aggregation: Trajectory Memory Retrieval Network for Video Object Segmentation
Memory-based methods in semi-supervised video object segmentation task achieve competitive performance by performing dense matching between query and memory frames. However, most of the existing methods neglect the f…
RetrievalSemantic SegmentationSemi-Supervised Video Object SegmentationVideo Object Segmentation+1Explore Synergistic Interaction Across Frames for Interactive Video Object Segmentation
Interactive Video Object Segmentation (iVOS) is a challenging task that requires real-time human-computer interaction. To improve the user experience, it is important to consider the user's input habits, segmentation qua…
Interactive Video Object SegmentationSemantic SegmentationVideo Object SegmentationVideo Semantic SegmentationFOMTrace: Interactive Video Segmentation By Image Graphs and Fuzzy Object Models
Common users have changed from mere consumers to active producers of multimedia data content. Video editing plays an important role in this scenario, calling for simple segmentation tools that can handle fast-moving and …
ObjectObject TrackingSegmentationVideo Editing+2SAM-MT: Real-Time Interactive Multi-Target Video Segmentation
Modern Video Object Segmentation (VOS) involves tracking and segmenting user-specified targets. While recent approaches have achieved remarkable performance in single-target scenarios, extending them to multi-target sett…
Video Object SegmentationVideo Segmentation