Visual Graph Memory With Unsupervised Representation for Visual Navigation
We present a novel graph-structured memory for visual navigation, called visual graph memory (VGM), which consists of unsupervised image representations obtained from navigation history. The proposed VGM is constructed incrementally based on the similarities among the unsupervised representations of observed images, and these representations are learned from an unlabeled image dataset. We also propose a navigation framework that can utilize the proposed VGM to tackle visual navigation problems. By incorporating a graph convolutional network and the attention mechanism, the proposed agent refers to the VGM to navigate the environment while simultaneously building the VGM. Using the VGM, the agent can embed its navigation history and other useful task-related information. We validate our approach on the visual navigation tasks using the Habitat simulator with the Gibson dataset, which provides a photo-realistic simulation environment. The extensive experimental results show that the proposed navigation agent with VGM surpasses the state-of-the-art approaches on image-goal navigation tasks.
Code (1)
Tasks
NavigateVisual NavigationSimilar Papers 제목 키워드 기반
Multimodal Neural Graph Memory Networks for Visual Question Answering
We introduce a new neural network architecture, Multimodal Neural Graph Memory Networks (MN-GMN), for visual question answering. The MN-GMN uses graph structure with different region features as node attributes and appli…
Graph Neural NetworkQuestion AnsweringVisual Question AnsweringVisual Question Answering (VQA)Memory Efficient Temporal & Visual Graph Model for Unsupervised Video Domain Adaptation
Existing video domain adaption (DA) methods need to store all temporal combinations of video frames or pair the source and target videos, which are memory cost expensive and can't scale up to long videos. To address thes…
Domain AdaptationGraph AttentionPushing the Limits of Fewshot Anomaly Detection in Industry Vision: Graphcore
In the area of fewshot anomaly detection (FSAD), efficient visual feature plays an essential role in memory bank M-based methods. However, these methods do not account for the relationship between the visual feature and …
Anomaly DetectionExploiting Structural Consistency of Chest Anatomy for Unsupervised Anomaly Detection in Radiography Images
Radiography imaging protocols focus on particular body regions, therefore producing images of great similarity and yielding recurrent anatomical structures across patients. Exploiting this structured information could po…
AnatomyAnomaly DetectionImage ReconstructionUnsupervised Anomaly DetectionJointly Visual- and Semantic-Aware Graph Memory Networks for Temporal Sentence Localization in Videos
Temporal sentence localization in videos (TSLV) aims to retrieve the most interested segment in an untrimmed video according to a given sentence query. However, almost of existing TSLV approaches suffer from the same lim…
Representation LearningSentenceVisual Reasoning