paper-with-me

홈 › Papers

Self-Monitoring Navigation Agent via Auxiliary Progress Estimation

2019-01-10 · ICLR 2019 5 · Chih-Yao Ma, Jiasen Lu, Zuxuan Wu, Ghassan AlRegib, Zsolt Kira, Richard Socher, Caiming Xiong

The Vision-and-Language Navigation (VLN) task entails an agent following navigational instruction in photo-realistic unknown environments. This challenging task demands that the agent be aware of which instruction was completed, which instruction is needed next, which way to go, and its navigation progress towards the goal. In this paper, we introduce a self-monitoring agent with two complementary components: (1) visual-textual co-grounding module to locate the instruction completed in the past, the instruction required for the next action, and the next moving direction from surrounding images and (2) progress monitor to ensure the grounded instruction correctly reflects the navigation progress. We test our self-monitoring agent on a standard benchmark and analyze our proposed approach through a series of ablation studies that elucidate the contributions of the primary components. Using our proposed method, we set the new state of the art by a significant margin (8% absolute increase in success rate on the unseen test set). Code is available at https://github.com/chihyaoma/selfmonitoring-agent .

📄 PDF Abstract BibTeX arXiv:1901.03035

Code (2)

chihyaoma/selfmonitoring-agent 공식 구현 pytorch
ayusefi/Localization-Papers

Tasks

Natural Language Visual GroundingVision and Language NavigationVision-Language NavigationVisual Navigation

Similar Papers 제목 키워드 기반

Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Tasks

2019-11-18 · CVPR 2020 6 · Fengda Zhu, Yi Zhu, Xiaojun Chang, Xiaodan Liang

Vision-Language Navigation (VLN) is a task where agents learn to navigate following natural language instructions. The key to this task is to perceive both the visual scene and natural language sequentially. Conventional…

NavigateVision-Language Navigation

Realistic PointGoal Navigation via Auxiliary Losses and Information Bottleneck

2021-09-17 · Guillermo Grande, Dhruv Batra, Erik Wijmans

We propose a novel architecture and training paradigm for training realistic PointGoal Navigation -- navigating to a target coordinate in an unseen environment under actuation and sensor noise without access to ground-tr…

PointGoal Navigation

ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI

2026-07-05 · ACE-Brain Team, :, Ziyang Gong, Haoming Gu 외 arxiv

Embodied AI is moving from isolated perception or action modules toward physical agents that understand, plan under goals, act through robot bodies, monitor progress, and improve from experience. Existing systems address…

Spatial ReasoningDecision Making

Auxiliary Tasks Speed Up Learning PointGoal Navigation

2020-07-09 · Joel Ye, Dhruv Batra, Erik Wijmans, Abhishek Das

PointGoal Navigation is an embodied task that requires agents to navigate to a specified point in an unseen environment. Wijmans et al. showed that this task is solvable but their method is computationally prohibitive, r…

GPUNavigatePointGoal Navigation

A Self-Supervised Auxiliary Loss for Deep RL in Partially Observable Settings

2021-04-17 · Eltayeb Ahmed, Luisa Zintgraf, Christian A. Schroeder de Witt, Nicolas Usunier

In this work we explore an auxiliary loss useful for reinforcement learning in environments where strong performing agents are required to be able to navigate a spatial environment. The auxiliary loss proposed is to mini…

NavigateSpatial Reasoning