Learning to See Through Obstructions with Layered Decomposition
We present a learning-based approach for removing unwanted obstructions, such as window reflections, fence occlusions, or adherent raindrops, from a short sequence of images captured by a moving camera. Our method leverages motion differences between the background and obstructing elements to recover both layers. Specifically, we alternate between estimating dense optical flow fields of the two layers and reconstructing each layer from the flow-warped images via a deep convolutional neural network. This learning-based layer reconstruction module facilitates accommodating potential errors in the flow estimation and brittle assumptions, such as brightness consistency. We show that the proposed approach learned from synthetically generated data performs well to real images. Experimental results on numerous challenging scenarios of reflection and fence removal demonstrate the effectiveness of the proposed method.
Code (1)
Tasks
Optical Flow EstimationSimilar Papers 제목 키워드 기반
LayerEdit: Disentangled Multi-Object Editing via Conflict-Aware Multi-Layer Learning
Text-driven multi-object image editing which aims to precisely modify multiple objects within an image based on text descriptions, has recently attracted considerable interest. Existing works primarily follow the localiz…
Object LocalizationImage EditingOmniPSD: Layered PSD Generation with Diffusion Transformer
Recent advances in diffusion models have greatly improved image generation and editing, yet generating or reconstructing layered PSD files with transparent alpha channels remains highly challenging. We propose OmniPSD, a…
Image GenerationKnowledge Discovery from Layered Neural Networks based on Non-negative Task Decomposition
Interpretability has become an important issue in the machine learning field, along with the success of layered neural networks in various practical tasks. Since a trained layered neural network consists of a complex non…
Certified Mapper: Repeated testing for acyclicity and obstructions to the nerve lemma
The Mapper algorithm does not include a check for whether the cover produced conforms to the requirements of the nerve lemma. To perform a check for obstructions to the nerve lemma, statistical considerations of multiple…
LEMMAExplicit Layer Modeling for Video Object Insertion and Layer Decomposition
Most video editing systems still lack explicit layered video representations, limiting their ability to perform realistic compositing, object reuse, and consistent manipulation. This limitation is especially pronounced i…