A Neural-Symbolic Architecture for Inverse Graphics Improved by Lifelong Meta-Learning
We follow the idea of formulating vision as inverse graphics and propose a new type of element for this task, a neural-symbolic capsule. It is capable of de-rendering a scene into semantic information feed-forward, as well as rendering it feed-backward. An initial set of capsules for graphical primitives is obtained from a generative grammar and connected into a full capsule network. Lifelong meta-learning continuously improves this network's detection capabilities by adding capsules for new and more complex objects it detects in a scene using few-shot learning. Preliminary results demonstrate the potential of our novel approach.
Code (1)
Tasks
Few-Shot LearningMeta-LearningSimilar Papers 제목 키워드 기반
A Neural-Symbolic Framework for Mental Simulation
We present a neural-symbolic framework for observing the environment and continuously learning visual semantics and intuitive physics to reproduce them in an interactive simulation. The framework consists of five parts, …
Meta-LearningNeuro-Logic Lifelong Learning
Solving Inductive Logic Programming (ILP) problems with neural networks is a key challenge in Neural-Symbolic Ar- tificial Intelligence (AI). While most research has focused on designing novel network architectures for i…
Inductive logic programmingVision-as-Inverse-Graphics Agent via Interleaved Multimodal Reasoning
Vision-as-inverse-graphics, the concept of reconstructing images into editable programs, remains challenging for Vision-Language Models (VLMs), which inherently lack fine-grained spatial grounding in one-shot settings. T…
Multimodal Reasoning3D ReconstructionBringing NeRFs to the Latent Space: Inverse Graphics Autoencoder
While pre-trained image autoencoders are increasingly utilized in computer vision, the application of inverse graphics in 2D latent spaces has been under-explored. Yet, besides reducing the training and rendering complex…
3D geometryNeRFHOUDINI: Lifelong Learning as Program Synthesis
We present a neurosymbolic framework for the lifelong learning of algorithmic tasks that mix perception and procedural reasoning. Reusing high-level concepts across domains and learning complex procedures are key challen…
Lifelong learningProgram SynthesisTransfer Learning