Rendition: Reclaiming what a black box takes away
The premise of our work is deceptively familiar: A black box $f(\cdot)$ has altered an image $\mathbf{x} \rightarrow f(\mathbf{x})$. Recover the image $\mathbf{x}$. This black box might be any number of simple or complicated things: a linear or non-linear filter, some app on your phone, etc. The latter is a good canonical example for the problem we address: Given only "the app" and an image produced by the app, find the image that was fed to the app. You can run the given image (or any other image) through the app as many times as you like, but you can not look inside the (code for the) app to see how it works. At first blush, the problem sounds a lot like a standard inverse problem, but it is not in the following sense: While we have access to the black box $f(\cdot)$ and can run any image through it and observe the output, we do not know how the block box alters the image. Therefore we have no explicit form or model of $f(\cdot)$. Nor are we necessarily interested in the internal workings of the black box. We are simply happy to reverse its effect on a particular image, to whatever extent possible. This is what we call the "rendition" (rather than restoration) problem, as it does not fit the mold of an inverse problem (blind or otherwise). We describe general conditions under which rendition is possible, and provide a remarkably simple algorithm that works for both contractive and expansive black box operators. The principal and novel take-away message from our work is this surprising fact: One simple algorithm can reliably undo a wide class of (not too violent) image distortions. A higher quality pdf of this paper is available at http://www.milanfar.org
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
CRISPnet: Color Rendition ISP Net
Image signal processors (ISPs) are historically grown legacy software systems for reconstructing color images from noisy raw sensor measurements. They are usually composited of many heuristic blocks for denoising, demosa…
DemosaickingDenoisingimage-classificationImage ClassificationAgency Among Agents: Designing with Hypertextual Friction in the Algorithmic Web
Today's algorithm-driven interfaces, from recommendation feeds to GenAI tools, often prioritize engagement and efficiency at the expense of user agency. As systems take on more decision-making, users have less control ov…
Aesthetics and neural network image representations
We analyze the spaces of images encoded by generative neural networks of the BigGAN architecture. We find that generic multiplicative perturbations of neural network parameters away from the photo-realistic point often l…
MetaISP -- Exploiting Global Scene Structure for Accurate Multi-Device Color Rendition
Image signal processors (ISPs) are historically grown legacy software systems for reconstructing color images from noisy raw sensor measurements. Each smartphone manufacturer has developed its ISPs with its own character…
Black-box optimization of noisy functions with unknown smoothness
We study the problem of black-box optimization of a function f of any dimension, given function evaluations perturbed by noise. The function is assumed to be locally smooth around one of its global optima, but this smoot…