paper-with-me

홈 › Papers

Learning to Compose: Improving Object Centric Learning by Injecting Compositionality

2024-05-01 · Whie Jung, Jaehoon Yoo, Sungjin Ahn, Seunghoon Hong

Learning compositional representation is a key aspect of object-centric learning as it enables flexible systematic generalization and supports complex visual reasoning. However, most of the existing approaches rely on auto-encoding objective, while the compositionality is implicitly imposed by the architectural or algorithmic bias in the encoder. This misalignment between auto-encoding objective and learning compositionality often results in failure of capturing meaningful object representations. In this study, we propose a novel objective that explicitly encourages compositionality of the representations. Built upon the existing object-centric learning framework (e.g., slot attention), our method incorporates additional constraints that an arbitrary mixture of object representations from two images should be valid by maximizing the likelihood of the composite data. We demonstrate that incorporating our objective to the existing framework consistently improves the objective-centric learning and enhances the robustness to the architectural choices.

📄 PDF Abstract BibTeX arXiv:2405.00646

Code (1)

whieya/learning-to-compose 공식 구현 jax

Tasks

ObjectSystematic GeneralizationvalidVisual Reasoning

Similar Papers 제목 키워드 기반

Illiterate DALL$\cdot$E Learns to Compose

2021-09-29 · ICLR 2022 4 · Gautam Singh, Fei Deng, Sungjin Ahn

DALL$\cdot$E has shown an impressive ability of composition-based systematic generalization in image generation. This is possible because it utilizes the dataset of text-image pairs where the text provides the source of …

DecoderImage GenerationSystematic Generalization

ROOTS: Object-Centric Representation and Rendering of 3D Scenes

2020-06-11 · Chang Chen, Fei Deng, Sungjin Ahn

A crucial ability of human intelligence is to build up models of individual 3D objects from partial scene observations. Recent works achieve object-centric generation but without the ability to infer the representation, …

ObjectRepresentation LearningScene Generation

Provably Learning Object-Centric Representations

2023-05-23 · Jack Brady, Roland S. Zimmermann, Yash Sharma, Bernhard Schölkopf 외

Learning structured representations of the visual world in terms of objects promises to significantly improve the generalization abilities of current machine learning models. While recent efforts to this end have shown p…

ObjectRepresentation Learning

Illiterate DALL-E Learns to Compose

2021-10-17 · Gautam Singh, Fei Deng, Sungjin Ahn

Although DALL-E has shown an impressive ability of composition-based systematic generalization in image generation, it requires the dataset of text-image pairs and the compositionality is provided by the text. In contras…

DecoderImage GenerationObjectSystematic Generalization

COLA: A Benchmark for Compositional Text-to-image Retrieval

2023-05-05 · NeurIPS 2023 11 · Arijit Ray, Filip Radenovic, Abhimanyu Dubey, Bryan A. Plummer 외

Compositional reasoning is a hallmark of human visual intelligence. Yet, despite the size of large vision-language models, they struggle to represent simple compositions by combining objects with their attributes. To mea…

AttributeCoLAImage RetrievalRetrieval