Perception-Distortion Trade-off in the SR Space Spanned by Flow Models
Flow-based generative super-resolution (SR) models learn to produce a diverse set of feasible SR solutions, called the SR space. Diversity of SR solutions increases with the temperature ($\tau$) of latent variables, which introduces random variations of texture among sample solutions, resulting in visual artifacts and low fidelity. In this paper, we present a simple but effective image ensembling/fusion approach to obtain a single SR image eliminating random artifacts and improving fidelity without significantly compromising perceptual quality. We achieve this by benefiting from a diverse set of feasible photo-realistic solutions in the SR space spanned by flow models. We propose different image ensembling and fusion strategies which offer multiple paths to move sample solutions in the SR space to more desired destinations in the perception-distortion plane in a controllable manner depending on the fidelity vs. perceptual quality requirements of the task at hand. Experimental results demonstrate that our image ensembling/fusion strategy achieves more promising perception-distortion trade-off compared to sample SR images produced by flow models and adversarially trained models in terms of both quantitative metrics and visual quality.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversitySuper-ResolutionSimilar Papers 제목 키워드 기반
Latent Posterior-Mean Rectified Flow for Higher-Fidelity Perceptual Face Restoration
The Perception-Distortion tradeoff (PD-tradeoff) theory suggests that face restoration algorithms must balance perceptual quality and fidelity. To achieve minimal distortion while maintaining perfect perceptual quality, …
Blind Face RestorationSpatio-Temporal Perception-Distortion Trade-off in Learned Video SR
Perception-distortion trade-off is well-understood for single-image super-resolution. However, its extension to video super-resolution (VSR) is not straightforward, since popular perceptual measures only evaluate natural…
Image Super-ResolutionOptical Flow EstimationSuper-ResolutionVideo Super-ResolutionA Theory of the Distortion-Perception Tradeoff in Wasserstein Space
The lower the distortion of an estimator, the more the distribution of its outputs generally deviates from the distribution of the signals it attempts to estimate. This phenomenon, known as the perception-distortion trad…
FormImage RestorationOpen-Ended Question AnsweringOn the Computation of the Gaussian Rate-Distortion-Perception Function
In this paper, we study the computation of the rate-distortion-perception function (RDPF) for a multivariate Gaussian source under mean squared error (MSE) distortion and, respectively, Kullback-Leibler divergence, geome…
Training-Free Rate-Distortion-Perception Traversal With Diffusion
The rate-distortion-perception (RDP) tradeoff characterizes the fundamental limits of lossy compression by jointly considering bitrate, reconstruction fidelity, and perceptual quality. While recent neural compression met…