PICS: Pairwise Image Compositing with Spatial Interactions
Despite strong single-turn performance, diffusion-based image compositing often struggles to preserve coherent spatial relations in pairwise or sequential edits, where subsequent insertions may overwrite previously generated content and disrupt physical consistency. We introduce PICS, a self-supervised composition-by-decomposition paradigm that composes objects in parallel while explicitly modeling the compositional interactions among (fully-/partially-)visible objects and background. At its core, an Interaction Transformer employs mask-guided Mixture-of-Experts to route background, exclusive, and overlap regions to dedicated experts, with an adaptive α-blending strategy that infers a compatibility-aware fusion of overlapping objects while preserving boundary fidelity. To further enhance robustness to geometric variations, we incorporate geometry-aware augmentations covering both out-of-plane and in-plane pose changes of objects. Our method delivers superior pairwise compositing quality and substantially improved stability, with extensive evaluations across virtual try-on, indoor, and street scene settings showing consistent gains over state-of-the-art baselines. Code and data are available at https://github.com/RyanHangZhou/PICS
Code (0)
등록된 구현이 없습니다.
Tasks
Virtual Try-onSimilar Papers 제목 키워드 기반
BIFRÖST: 3D-Aware Image compositing with Language Instructions
This paper introduces Bifr\"ost, a novel 3D-aware framework that is built upon diffusion models to perform instruction-based image composition. Previous methods concentrate on image compositing at the 2D level, which fal…
counterfactualImage HarmonizationMISC: Multi-Condition Injection and Spatially-Adaptive Compositing for Conditional Person Image Synthesis
In this paper, we explore synthesizing person images with multiple conditions for various backgrounds. To this end, we propose a framework named "MISC" for conditional image generation and image compositing. For conditio…
Conditional Image GenerationImage GenerationChameleon: Style-Content Disentangled Framework for Cross-Domain Object Compositing
Image compositing aims to seamlessly insert a foreground object into a background image, and recent advances in diffusion models have significantly enhanced the quality, especially when the foreground and background imag…
Contrastive LearningGo-with-the-Track: Video Compositing and Motion Control with Point Tracking
Filmmaking demands precise motion control and reference image compositing -- capabilities that existing methods treat separately. Point-track-conditioned image-to-video models restrict content insertion to the first fram…
Video GenerationPoint TrackingYour Latent Mask is Wrong: Pixel-Equivalent Latent Compositing for Diffusion Models
Latent inpainting in diffusion models still relies almost universally on linearly interpolating VAE latents under a downsampled mask. We propose a key principle for compositing image latents: Pixel-Equivalent Latent Comp…