Point-wise mutual information-based video segmentation with high temporal consistency
In this paper, we tackle the problem of temporally consistent boundary detection and hierarchical segmentation in videos. While finding the best high-level reasoning of region assignments in videos is the focus of much recent research, temporal consistency in boundary detection has so far only rarely been tackled. We argue that temporally consistent boundaries are a key component to temporally consistent region assignment. The proposed method is based on the point-wise mutual information (PMI) of spatio-temporal voxels. Temporal consistency is established by an evaluation of PMI-based point affinities in the spectral domain over space and time. Thus, the proposed method is independent of any optical flow computation or previously learned motion models. The proposed low-level video segmentation method outperforms the learning-based state of the art in terms of standard region metrics.
Code (0)
등록된 구현이 없습니다.
Tasks
Boundary DetectionOptical Flow EstimationSegmentationVideo SegmentationVideo Semantic SegmentationVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Training-Free Robust Interactive Video Object Segmentation
Interactive video object segmentation is a crucial video task, having various applications from video editing to data annotating. However, current approaches struggle to accurately segment objects across diverse domains.…
Interactive Video Object SegmentationObjectPoint TrackingSegmentation+5Region Mutual Information Loss for Semantic Segmentation
Semantic segmentation is a fundamental problem in computer vision. It is considered as a pixel-wise classification problem in practice, and most segmentation models use a pixel-wise loss as their optimization riterion. H…
Semantic SegmentationOn the Properties and Estimation of Pointwise Mutual Information Profiles
The pointwise mutual information profile, or simply profile, is the distribution of pointwise mutual information for a given pair of random variables. One of its important properties is that its expected value is precise…
Mutual Information EstimationUncertainty QuantificationVideo Action Segmentation via Contextually Refined Temporal Keypoints
Video action segmentation refers to the task of densely casting each video frame or short segment in an untrimmed video into some pre-specified action categories. Although recent years have witnessed a great promise …
Action SegmentationGraph MatchingSegmentationAdversarial Mutual Leakage Network for Cell Image Segmentation
We propose three segmentation methods using GAN and information leakage between generator and discriminator. First, we propose an Adversarial Training Attention Module (ATA-Module) that uses an attention mechanism from t…
Image SegmentationSegmentationSemantic Segmentation