Compositional GAN (Extended Abstract): Learning Image-Conditional Binary Composition
Generative Adversarial Networks (GANs) can produce images of surprising complexity and realism but are generally structured to sample from a single latent source ignoring the explicit spatial interaction between multiple entities that could be present in a scene. Capturing such complex interactions between different objects in the world, including their relative scaling, spatial layout, occlusion, or viewpoint transformation is a challenging problem. In this work, we compose a pair of objects in a conditional GAN framework using a novel self-consistent composition-by-decomposition network. Given object images from two distinct distributions, our model can generate a realistic composite image from their joint distribution following the texture and shape of the input objects. Our results reveal that the learned model captures potential interactions between the two object domains, and can output their realistic composed scene at test time.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Language as an Abstraction for Hierarchical Deep Reinforcement Learning
Solving complex, temporally-extended tasks is a long-standing problem in reinforcement learning (RL). We hypothesize that one critical element of solving such problems is the notion of compositionality. With the ability …
Deep Reinforcement LearningInstruction FollowingMuJoCoObject+4Local Mechanisms of Compositional Generalization in Conditional Diffusion
Conditional diffusion models appear capable of compositional generalization, i.e., generating convincing samples for out-of-distribution combinations of conditioners, but the mechanisms underlying this ability remain unc…
Structured Active Inference (Extended Abstract)
We introduce structured active inference, a large generalization and formalization of active inference using the tools of categorical systems theory. We cast generative models formally as systems "on an interface", with …
Yin and Yang: Balancing and Answering Binary Visual Questions
The complex compositional structure of language makes problems at the intersection of vision and language challenging. But language also provides a strong prior that can result in good superficial performance, without th…
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)Prism: A Minimal Compositional Metalanguage for Specifying Agent Behavior
Prism is a small, compositional metalanguage for specifying the behaviour of tool-using software agents. Rather than introducing ad hoc control constructs, Prism is built around a fixed core context, Core1, which provide…