Memory Selection Network for Video Propagation
Video propagation is a fundamental problem in video processing where guidance frame predictions are propagated to guide predictions of the target frame. Previous research mainly treats the previous adjacent frame as guidance, which, however, could make the propagation vulnerable to occlusion, large motion, and inaccurate information in the previous adjacent frame. To tackle this challenge, we propose a memory selection network, which learns to select suitable guidance from all previous frames for effective and robust propagation. Experimental results on video object segmentation and video colorization tasks show that our method consistently improves performance and can robustly handle challenging scenarios in video propagation.
Code (0)
등록된 구현이 없습니다.
Tasks
ColorizationSemantic SegmentationVideo Object SegmentationVideo PropagationVideo Semantic SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Stochastic Backpropagation: A Memory Efficient Strategy for Training Video Models
We propose a memory efficient method, named Stochastic Backpropagation (SBP), for training deep neural networks on videos. It is based on the finding that gradients from incomplete execution for backpropagation can still…
Action DetectionAction RecognitionGPUChopGrad: Pixel-Wise Losses for Latent Video Diffusion via Truncated Backpropagation
Recent video diffusion models achieve high-quality generation through recurrent frame processing where each frame generation depends on previous frames. However, this recurrent mechanism means that training such models i…
Video Super-ResolutionVideo EnhancementVideo GenerationVideo InpaintingFast SAM2 with Text-Driven Token Pruning
Segment Anything Model 2 (SAM2), a vision foundation model has significantly advanced in prompt-driven video object segmentation, yet their practical deployment remains limited by the high computational and memory cost o…
Video Object SegmentationVideo SegmentationMatAnyone: Stable Video Matting with Consistent Memory Propagation
Auxiliary-free human video matting methods, which rely solely on input frames, often struggle with complex or ambiguous backgrounds. To address this, we propose MatAnyone, a robust framework tailored for target-assigned …
Image MattingVideo MattingOmniMem: Scalable and Adaptive Memory Retrieval for Long Video Generation
Autoregressive (AR) video generation extends videos by producing latent chunks sequentially, but scaling to long videos requires repeated access to a growing historical KV cache. Existing methods reduce this cost by trun…
Video Generation