paper-with-me

홈 › Papers

SinMPI: Novel View Synthesis from a Single Image with Expanded Multiplane Images

2023-12-18 · Guo Pu, Peng-Shuai Wang, Zhouhui Lian

Single-image novel view synthesis is a challenging and ongoing problem that aims to generate an infinite number of consistent views from a single input image. Although significant efforts have been made to advance the quality of generated novel views, less attention has been paid to the expansion of the underlying scene representation, which is crucial to the generation of realistic novel view images. This paper proposes SinMPI, a novel method that uses an expanded multiplane image (MPI) as the 3D scene representation to significantly expand the perspective range of MPI and generate high-quality novel views from a large multiplane space. The key idea of our method is to use Stable Diffusion to generate out-of-view contents, project all scene contents into an expanded multiplane image according to depths predicted by monocular depth estimators, and then optimize the multiplane image under the supervision of pseudo multi-view data generated by a depth-aware warping and inpainting module. Both qualitative and quantitative experiments have been conducted to validate the superiority of our method to the state of the art. Our code and data are available at https://github.com/TrickyGo/SinMPI.

📄 PDF Abstract BibTeX arXiv:2312.11037

Code (1)

trickygo/sinmpi 공식 구현 pytorch

Tasks

Novel View Synthesis

Methods 이 논문이 사용한 방법론

Inpainting Train a convolutional neural network to generate the contents of an arbitrary image region conditioned on its surroundings.
Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Similar Papers 제목 키워드 기반

Single-View View Synthesis with Multiplane Images

2020-04-23 · CVPR 2020 6 · Richard Tucker, Noah Snavely

A recent strand of work in view synthesis uses deep learning to generate multiplane images (a camera-centric, layered 3D representation) given two or more input images at known viewpoints. We apply this representation to…

Single-View View Synthesis with Self-Rectified Pseudo-Stereo

2023-04-19 · Yang Zhou, Hanjie Wu, Wenxi Liu, Zheng Xiong 외

Synthesizing novel views from a single view image is a highly ill-posed problem. We discover an effective solution to reduce the learning ambiguity by expanding the single-view view synthesis problem to a multi-view sett…

3D ReconstructionNovel View Synthesis

HumanGif: Single-View Human Diffusion with Generative Prior

2025-02-17 · Shoukang Hu, Takuya Narihira, Kazumi Fukuda, Ryosuke Sawata 외

While previous single-view-based 3D human reconstruction methods made significant progress in novel view synthesis, it remains a challenge to synthesize both view-consistent and pose-consistent results for animatable hum…

3D Human ReconstructionNeRFNovel View Synthesis

When Single Answer Is Not Enough: Rethinking Single-Step Retrosynthesis Benchmarks for LLMs

2026-02-03 · Bogdan Zagribelnyy, Ivan Ilin, Maksim Kuznetsov, Nikita Bondarev 외 arxiv

Recent progress has expanded the use of large language models (LLMs) in drug discovery, including synthesis planning. However, objective evaluation of retrosynthesis performance remains limited. Existing benchmarks and m…

Single-step retrosynthesisDrug Discovery

Learning Partonomic 3D Reconstruction from Image Collections

2025-01-01 · CVPR 2025 1 · Xiaoqian Ruan, Pei Yu, Dian Jia, Hyeonjeong Park 외

Reconstructing the 3D shape of an object from a single-view image is a fundamental task in computer vision. Recent advances in differentiable rendering have enabled 3D reconstruction from image collections using only…

3D ReconstructionImage GenerationObjectObject Reconstruction