Bayesian Relational Memory for Semantic Visual Navigation
We introduce a new memory architecture, Bayesian Relational Memory (BRM), to improve the generalization ability for semantic visual navigation agents in unseen environments, where an agent is given a semantic target to navigate towards. BRM takes the form of a probabilistic relation graph over semantic entities (e.g., room types), which allows (1) capturing the layout prior from training environments, i.e., prior knowledge, (2) estimating posterior layout at test time, i.e., memory update, and (3) efficient planning for navigation, altogether. We develop a BRM agent consisting of a BRM module for producing sub-goals and a goal-conditioned locomotion module for control. When testing in unseen environments, the BRM agent outperforms baselines that do not explicitly utilize the probabilistic relational memory structure
Code (1)
Tasks
NavigateVisual NavigationSimilar Papers 제목 키워드 기반
HAIR: Hierarchical Visual-Semantic Relational Reasoning for Video Question Answering
Relational reasoning is at the heart of video question answering. However, existing approaches suffer from several common limitations: (1) they only focus on either object-level or frame-level relational reasoning, a…
Question AnsweringRelational ReasoningVideo Question AnsweringSemantic Evidence Regulation via Relational Bias for Zero-Shot Object Navigation
Object navigation requires an embodied agent to locate a target object in an unknown environment through visual observations. Existing zero-shot methods typically leverage open-vocabulary perception and semantic priors t…
Entropy in Semantic Memory Navigation in Blind and Sighted Individuals: The Effect of Visual Experience
Embodied accounts of semantic memory highlight the role of sensorimotor systems in acquiring and storing knowledge. Congenitally blind populations offer a critical test bed for these assumptions, providing an opportunity…
Spatial Relation Graph and Graph Convolutional Network for Object Goal Navigation
This paper describes a framework for the object-goal navigation task, which requires a robot to find and move to the closest instance of a target object class from a random starting position. The framework uses a history…
Bayesian InferenceObjectPositionRelationSemantic Audio-Visual Navigation
Recent work on audio-visual navigation assumes a constantly-sounding target and restricts the role of audio to signaling the target's position. We introduce semantic audio-visual navigation, where objects in the environm…
PositionVisual Navigation