Transformer-based Localization from Embodied Dialog with Large-scale Pre-training
We address the challenging task of Localization via Embodied Dialog (LED). Given a dialog from two agents, an Observer navigating through an unknown environment and a Locator who is attempting to identify the Observer's location, the goal is to predict the Observer's final location in a map. We develop a novel LED-Bert architecture and present an effective pretraining strategy. We show that a graph-based scene representation is more effective than the top-down 2D maps used in prior works. Our approach outperforms previous baselines.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
DiaLoc: An Iterative Approach to Embodied Dialog Localization
Multimodal learning has advanced the performance for many vision-language tasks. However, most existing works in embodied dialog research focus on navigation and leave the localization task understudied. The few existing…
Where Are You? Localization from Embodied Dialog
We present Where Are You? (WAY), a dataset of ~6k dialogs in which two humans -- an Observer and a Locator -- complete a cooperative localization task. The Observer is spawned at random in a 3D environment and can naviga…
NavigateVisual DialogAdvancing DialNav through Automatic Embodied Dialog Augmentation
For embodied agents capable of physical interaction, the capability to create and understand dialog is crucial to ensure both safety and effectiveness. While DialNav~\cite{han2025dialnav} provides a framework for holisti…
Dialog Acts for Task-Driven Embodied Agents
Embodied agents need to be able to interact in natural language understanding task descriptions and asking appropriate follow up questions to obtain necessary information to be effective at successfully accomplishing tas…
Natural Language UnderstandingDialog Acts for Task Driven Embodied Agents
Embodied agents need to be able to interact in natural language – understanding task descriptions and asking appropriate follow up questions to obtain necessary information to be effective at successfully accomplishing t…
Natural Language Understanding