ProbFlow: Joint Optical Flow and Uncertainty Estimation
Optical flow estimation remains challenging due to untextured areas, motion boundaries, occlusions, and more. Thus, the estimated flow is not equally reliable across the image. To that end, post-hoc confidence measures have been introduced to assess the per-pixel reliability of the flow. We overcome the artificial separation of optical flow and confidence estimation by introducing a method that jointly predicts optical flow and its underlying uncertainty. Starting from common energy-based formulations, we rely on the corresponding posterior distribution of the flow given the images. We derive a variational inference scheme based on mean field, which incorporates best practices from energy minimization. An uncertainty measure is obtained along the flow at every pixel as the (marginal) entropy of the variational distribution. We demonstrate the flexibility of our probabilistic approach by applying it to two different energies and on two benchmarks. We not only obtain flow results that are competitive with the underlying energy minimization approach, but also a reliable uncertainty measure that significantly outperforms existing post-hoc approaches.
Code (0)
등록된 구현이 없습니다.
Tasks
Optical Flow EstimationVariational InferenceSimilar Papers 제목 키워드 기반
U$^{2}$Flow: Uncertainty-Aware Unsupervised Optical Flow Estimation
Unsupervised optical flow methods typically lack reliable uncertainty estimation, limiting their robustness and interpretability. We propose U$^{2}$Flow, the first recurrent unsupervised framework that jointly estimates …
Optical Flow EstimationJoint Self-supervised Depth and Optical Flow Estimation towards Dynamic Objects
Significant attention has been attracted to deep learning-based depth estimates. Dynamic objects become the most hard problems in inter-frame-supervised depth estimates due to the uncertainty in adjacent frames. Thus, in…
Depth EstimationMotion SegmentationOptical Flow EstimationUncertainty-Driven Dense Two-View Structure from Motion
This work introduces an effective and practical solution to the dense two-view structure from motion (SfM) problem. One vital question addressed is how to mindfully use per-pixel optical flow correspondence between two f…
Depth EstimationOptical Flow EstimationPose EstimationVocal Bursts Valence PredictionUncertainty Estimates and Multi-Hypotheses Networks for Optical Flow
Optical flow estimation can be formulated as an end-to-end supervised learning problem, which yields estimates with a superior accuracy-runtime tradeoff compared to alternative methodology. In this paper, we make such ne…
Optical Flow EstimationEvery Frame Counts: Joint Learning of Video Segmentation and Optical Flow
A major challenge for video semantic segmentation is the lack of labeled data. In most benchmark datasets, only one frame of a video clip is annotated, which makes most supervised methods fail to utilize information from…
Optical Flow EstimationSegmentationSemantic SegmentationVideo Segmentation+1