Learning Internal Representations of 3D Transformations from 2D Projected Inputs
When interacting in a three dimensional world, humans must estimate 3D structure from visual inputs projected down to two dimensional retinal images. It has been shown that humans use the persistence of object shape over motion-induced transformations as a cue to resolve depth ambiguity when solving this underconstrained problem. With the aim of understanding how biological vision systems may internally represent 3D transformations, we propose a computational model, based on a generative manifold model, which can be used to infer 3D structure from the motion of 2D points. Our model can also learn representations of the transformations with minimal supervision, providing a proof of concept for how humans may develop internal representations on a developmental or evolutionary time scale. Focused on rotational motion, we show how our model infers depth from moving 2D projected points, learns 3D rotational transformations from 2D training stimuli, and compares to human performance on psychophysical structure-from-motion experiments.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Jump to Conclusions: Short-Cutting Transformers With Linear Transformations
Transformer-based language models create hidden representations of their inputs at every layer, but only use final-layer representations for prediction. This obscures the internal decision-making process of the model and…
Decision MakingLanguage ModelingLanguage ModellingA model of cortical cognitive function using hierarchical interactions of gating matrices in internal agents coding relational representations
Flexible cognition requires the ability to rapidly detect systematic functions of variables and guide future behavior based on predictions. The model described here proposes a potential framework for patterns of neural a…
Improving the Robustness of Capsule Networks to Image Affine Transformations
Convolutional neural networks (CNNs) achieve translational invariance by using pooling operations. However, the operations do not preserve the spatial relationships in the learned representations. Hence, CNNs cannot extr…
SEIS: Subspace-based Equivariance and Invariance Scores for Neural Representations
Understanding how neural representations respond to geometric transformations is essential for evaluating whether learned features preserve meaningful spatial structure. Existing approaches primarily assess robustness pr…
Multi-Task LearningData AugmentationGated Word-Character Recurrent Language Model
We introduce a recurrent neural network language model (RNN-LM) with long short-term memory (LSTM) units that utilizes both character-level and word-level inputs. Our model has a gate that adaptively finds the optimal mi…
Language ModelingLanguage Modellingmodel