Explicitly Disentangled Representations in Object-Centric Learning
Extracting structured representations from raw visual data is an important and long-standing challenge in machine learning. Recently, techniques for unsupervised learning of object-centric representations have raised growing interest. In this context, enhancing the robustness of the latent features can improve the efficiency and effectiveness of the training of downstream tasks. A promising step in this direction is to disentangle the factors that cause variation in the data. Previously, Invariant Slot Attention disentangled position, scale, and orientation from the remaining features. Extending this approach, we focus on separating the shape and texture components. In particular, we propose a novel architecture that biases object-centric models toward disentangling shape and texture components into two non-overlapping subsets of the latent space dimensions. These subsets are known a priori, hence before the training process. Experiments on a range of object-centric benchmarks reveal that our approach achieves the desired disentanglement while also numerically improving baseline performance in most cases. In addition, we show that our method can generate novel textures for a specific object or transfer textures between objects with distinct shapes.
Code (1)
Tasks
DisentanglementObjectMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
When Object-Centric World Models Meet Policy Learning: From Pixels to Policies, and Where It Breaks
Object-centric world models (OCWM) aim to decompose visual scenes into object-level representations, providing structured abstractions that could improve compositional generalization and data efficiency in reinforcement …
Reinforcement LearningLearning Global Object-Centric Representations via Disentangled Slot Attention
Humans can discern scene-independent features of objects across various environments, allowing them to swiftly identify objects amidst changing factors such as lighting, perspective, size, and position and imagine the co…
ObjectPositionRepresentation LearningScene GenerationLearning Disentangled Representation in Object-Centric Models for Visual Dynamics Prediction via Transformers
Recent work has shown that object-centric representations can greatly help improve the accuracy of learning dynamics while also bringing interpretability. In this work, we take this idea one step further, ask the followi…
AttributeObjectDynamic Scene Understanding through Object-Centric Voxelization and Neural Rendering
Learning object-centric representations from unsupervised videos is challenging. Unlike most previous approaches that focus on decomposing 2D images, we present a 3D generative model named DynaVol-S for dynamic scenes th…
Inverse RenderingNeRFNeural RenderingNovel View Synthesis+3Language-Mediated, Object-Centric Representation Learning
We present Language-mediated, Object-centric Representation Learning (LORL), a paradigm for learning disentangled, object-centric scene representations from vision and language. LORL builds upon recent advances in unsupe…
ObjectObject DiscoveryReferring ExpressionReferring Expression Comprehension+3