Adaptive Mask-based Pyramid Network for Realistic Bokeh Rendering
Bokeh effect highlights an object (or any part of the image) while blurring the rest of the image, and creates a visually pleasant artistic effect. Due to the sensor-based limitations on mobile devices, machine learning (ML) based bokeh rendering has gained attention as a reliable alternative. In this paper, we focus on several improvements in ML-based bokeh rendering; i) on-device performance with high-resolution images, ii) ability to guide bokeh generation with user-editable masks and iii) ability to produce varying blur strength. To this end, we propose Adaptive Mask-based Pyramid Network (AMPN), which is formed of a Mask-Guided Bokeh Generator (MGBG) block and a Laplacian Pyramid Refinement (LPR) block. MGBG consists of two lightweight networks stacked to each other to generate the bokeh effect, and LPR refines and upsamples the output of MGBG to produce the high-resolution bokeh image. We achieve i) via our lightweight, mobile-friendly design choices, ii) via the stacked-network design of MGBG and the weakly-supervised mask prediction scheme and iii) via manually or automatically editing the intensity values of the mask that guide the bokeh generation. In addition to these features, our results show that AMPN produces competitive or better results compared to existing methods on the EBB! dataset, while being faster and smaller than the alternatives.
Code (0)
등록된 구현이 없습니다.
Tasks
Bokeh Effect RenderingSimilar Papers 제목 키워드 기반
Defocus to focus: Photo-realistic bokeh rendering by fusing defocus and radiance priors
We consider the problem of realistic bokeh rendering from a single all-in-focus image. Bokeh rendering mimics aesthetic shallow depth-of-field (DoF) in professional photography, but these visual effects generated by exis…
HallucinationTowards Photorealistic and Efficient Bokeh Rendering via Diffusion Framework
Existing mobile devices are constrained by compact optical designs, such as small apertures, which make it difficult to produce natural, optically realistic bokeh effects. Although recent learning-based methods have show…
Depth EstimationBokehlicious: Photorealistic Bokeh Rendering with Controllable Apertures
Bokeh rendering methods play a key role in creating the visually appealing, softly blurred backgrounds seen in professional photography. While recent learning-based approaches show promising results, generating realistic…
DeblurringZero-shot GeneralizationBGGAN: Bokeh-Glass Generative Adversarial Network for Rendering Realistic Bokeh
A photo captured with bokeh effect often means objects in focus are sharp while the out-of-focus areas are all blurred. DSLR can easily render this kind of effect naturally. However, due to the limitation of sensors, sma…
Bokeh Effect RenderingGenerative Adversarial NetworkGPUImage-to-Image TranslationGBSD: Generative Bokeh with Stage Diffusion
The bokeh effect is an artistic technique that blurs out-of-focus areas in a photograph and has gained interest due to recent developments in text-to-image synthesis and the ubiquity of smart-phone cameras and photo-shar…
Image GenerationImage ManipulationNeural Rendering