Differentiable Image Parameterizations
Typically, we parameterize the input image as the RGB values of each pixel, but that isn’t the only way. As long as the mapping from parameters to images is differentiable, we can still optimize alternative parameterizations with gradient descent. Differentiable image parameterizations invite us to ask “what kind of image generation process can we backpropagate through?” The answer is quite a lot, and some of the more exotic possibilities can create a wide range of interesting effects, including 3D neural art, images with transparency, and aligned interpolation. Previous work using specific unusual image parameterizations has shown exciting results — we think that zooming out and looking at this area as a whole suggests there’s even more potential.
Code (2)
Tasks
Image GenerationSimilar Papers 제목 키워드 기반
Joint Parameter and Parameterization Inference with Uncertainty Quantification through Differentiable Programming
Accurate representations of unknown and sub-grid physical processes through parameterizations (or closure) in numerical simulations with quantified uncertainty are critical for resolving the coarse-grained partial differ…
Bayesian InferenceUncertainty QuantificationAdjoint-based online learning of two-layer quasi-geostrophic baroclinic turbulence
For reasons of computational constraint, most global ocean circulation models used for Earth System Modeling still rely on parameterizations of sub-grid processes, and limitations in these parameterizations affect the mo…
Zero-Shot Function Encoder-Based Differentiable Predictive Control
We introduce a differentiable framework for zero-shot adaptive control over parametric families of nonlinear dynamical systems. Our approach integrates a function encoder-based neural ODE (FE-NODE) for modeling system dy…
Self-Supervised LearningVariational Entropic Optimal Transport
Entropic optimal transport (EOT) in continuous spaces with quadratic cost is a classical tool for solving the domain translation problem. In practice, recent approaches optimize a weak dual EOT objective depending on a s…
Image-to-Image TranslationFlowMap: High-Quality Camera Poses, Intrinsics, and Depth via Gradient Descent
This paper introduces FlowMap, an end-to-end differentiable method that solves for precise camera poses, camera intrinsics, and per-frame dense depth of a video sequence. Our method performs per-video gradient-descent mi…
Novel View SynthesisOptical Flow EstimationPoint Tracking