HindSight: A Graph-Based Vision Model Architecture For Representing Part-Whole Hierarchies
This paper presents a model architecture for encoding the representations of part-whole hierarchies in images in form of a graph. The idea is to divide the image into patches of different levels and then treat all of these patches as nodes for a fully connected graph. A dynamic feature extraction module is used to extract feature representations from these patches in each graph iteration. This enables us to learn a rich graph representation of the image that encompasses the inherent part-whole hierarchical information. Utilizing proper self-supervised training techniques, such a model can be trained as a general purpose vision encoder model which can then be used for various vision related downstream tasks (e.g., Image Classification, Object Detection, Image Captioning, etc.).
Code (0)
등록된 구현이 없습니다.
Tasks
Image Captioningimage-classificationImage Classificationobject-detectionObject DetectionSimilar Papers 제목 키워드 기반
TurnSight: Turn-Level Hindsight Self-Distillation for Tool-Integrated Reasoning
Tool-Integrated Reasoning (TIR) enables LLMs to solve complex tasks through iterative tool interactions. However, existing reinforcement learning methods often rely on trajectory-level supervision, limiting fine-grained …
Reinforcement LearningVisual Hindsight Self-Imitation Learning for Interactive Navigation
Interactive visual navigation tasks, which involve following instructions to reach and interact with specific targets, are challenging not only because successful experiences are very rare but also because the complex vi…
Imitation LearningVisual NavigationGUMBEL-NERF: Representing Unseen Objects as Part-Compositional Neural Radiance Fields
We propose Gumbel-NeRF, a mixture-of-expert (MoE) neural radiance fields (NeRF) model with a hindsight expert selection mechanism for synthesizing novel views of unseen objects. Previous studies have shown that the MoE s…
NeRFNovel View SynthesisComplex Robotic Manipulation via Graph-Based Hindsight Goal Generation
Reinforcement learning algorithms such as hindsight experience replay (HER) and hindsight goal generation (HGG) have been able to solve challenging robotic manipulation tasks in multi-goal settings with sparse rewards. H…
Dynamical-VAE-based Hindsight to Learn the Causal Dynamics of Factored-POMDPs
Learning representations of underlying environmental dynamics from partial observations is a critical challenge in machine learning. In the context of Partially Observable Markov Decision Processes (POMDPs), state repres…