VALAN: Vision and Language Agent Navigation
VALAN is a lightweight and scalable software framework for deep reinforcement learning based on the SEED RL architecture. The framework facilitates the development and evaluation of embodied agents for solving grounded language understanding tasks, such as Vision-and-Language Navigation and Vision-and-Dialog Navigation, in photo-realistic environments, such as Matterport3D and Google StreetView. We have added a minimal set of abstractions on top of SEED RL allowing us to generalize the architecture to solve a variety of other RL problems. In this article, we will describe VALAN's software abstraction and architecture, and also present an example of using VALAN to design agents for instruction-conditioned indoor navigation.
Code (1)
Tasks
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Vision and Language NavigationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Environment-agnostic Multitask Learning for Natural Language Grounded Navigation
Recent research efforts enable study for natural language grounded navigation in photo-realistic environments, e.g., following natural language instructions or dialog. However, existing methods tend to overfit training d…
Vision-Language NavigationFast Vision in the Dark: A Case for Single-Photon Imaging in Planetary Navigation
Improving robotic navigation is critical for extending exploration range and enhancing operational efficiency. Vision-based navigation relying on traditional CCD or CMOS cameras faces major challenges when complex illumi…
NavHint: Vision and Language Navigation Agent with a Hint Generator
Existing work on vision and language navigation mainly relies on navigation-related losses to establish the connection between vision and language modalities, neglecting aspects of helping the navigation agent build a de…
Vision and Language NavigationDiagnosing Vision-and-Language Navigation: What Really Matters
Vision-and-language navigation (VLN) is a multimodal task where an agent follows natural language instructions and navigates in visual environments. Multiple setups have been proposed, and researchers apply new model arc…
DiagnosticObjectVision and Language NavigationLanguage-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environments
In the Vision-and-Language Navigation (VLN) task an embodied agent navigates a 3D environment, following natural language instructions. A challenge in this task is how to handle 'off the path' scenarios where an agent ve…
Vision and Language Navigation