Video prediction using score-based conditional density estimation
Temporal prediction is inherently uncertain, but representing the ambiguity in natural image sequences is a challenging high-dimensional probabilistic inference problem. For natural scenes, the curse of dimensionality renders explicit density estimation statistically and computationally intractable. Here, we describe an implicit regression-based framework for learning and sampling the conditional density of the next frame in a video given previous observed frames. We show that sequence-to-image deep networks trained on a simple resilience-to-noise objective function extract adaptive representations for temporal prediction. Synthetic experiments demonstrate that this score-based framework can handle occlusion boundaries: unlike classical methods that average over bifurcating temporal trajectories, it chooses among likely trajectories, selecting more probable options with higher frequency. Furthermore, analysis of networks trained on natural image sequences reveals that the representation automatically weights predictive evidence by its reliability, which is a hallmark of statistical inference
Code (0)
등록된 구현이 없습니다.
Tasks
Density EstimationImage Deep NetworksPredictionVideo PredictionSimilar Papers 제목 키워드 기반
Neural-Kernelized Conditional Density Estimation
Conditional density estimation is a general framework for solving various problems in machine learning. Among existing methods, non-parametric and/or kernel-based methods are often difficult to use on large datasets, whi…
Density EstimationDimensionality ReductionRepresentation LearningParallel Multiscale Autoregressive Density Estimation
PixelCNN achieves state-of-the-art results in density estimation for natural images. Although training is fast, inference is costly, requiring one network evaluation per pixel; O(N) for N pixels. This can be sped up by c…
Conditional Image GenerationDensity EstimationImage CompressionImage Generation+1CONTRA: Conformal Prediction Region via Normalizing Flow Transformation
Density estimation and reliable prediction regions for outputs are crucial in supervised and unsupervised learning. While conformal prediction effectively generates coverage-guaranteed regions, it struggles with multi-di…
Density EstimationSplit Localized Conformal Prediction
Conformal prediction is a simple and powerful tool that can quantify uncertainty without any distributional assumptions. Many existing methods only address the average coverage guarantee, which is not ideal compared to t…
Conformal PredictionDensity EstimationPredictionPrediction Intervals+1Fast Nonparametric Conditional Density Estimation
Conditional density estimation generalizes regression by modeling a full density f(yjx) rather than only the expected value E(yjx). This is important for many tasks, including handling multi-modality and generating predi…
Density EstimationPrediction Intervals