paper-with-me

Papers

Transformer-based Localization from Embodied Dialog with Large-scale Pre-training

2022-10-10 · Meera Hahn, James M. Rehg

We address the challenging task of Localization via Embodied Dialog (LED). Given a dialog from two agents, an Observer navigating through an unknown environment and a Locator who is attempting to identify the Observer's location, the goal is to predict the Observer's final location in a map. We develop a novel LED-Bert architecture and present an effective pretraining strategy. We show that a graph-based scene representation is more effective than the top-down 2D maps used in prior works. Our approach outperforms previous baselines.

📄 PDF Abstract BibTeX arXiv:2210.04864

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

DiaLoc: An Iterative Approach to Embodied Dialog Localization

2024-03-11 · CVPR 2024 1 · Chao Zhang, Mohan Li, Ignas Budvytis, Stephan Liwicki

Multimodal learning has advanced the performance for many vision-language tasks. However, most existing works in embodied dialog research focus on navigation and leave the localization task understudied. The few existing…

Where Are You? Localization from Embodied Dialog

2020-11-16 · EMNLP 2020 11 · Meera Hahn, Jacob Krantz, Dhruv Batra, Devi Parikh 외

We present Where Are You? (WAY), a dataset of ~6k dialogs in which two humans -- an Observer and a Locator -- complete a cooperative localization task. The Observer is spawned at random in a 3D environment and can naviga…

NavigateVisual Dialog

Advancing DialNav through Automatic Embodied Dialog Augmentation

2026-06-18 · Leekyeung Han, Sangwon Jung, Hyunji Min, Jinseong Jeong 외 arxiv

For embodied agents capable of physical interaction, the capability to create and understand dialog is crucial to ensure both safety and effectiveness. While DialNav~\cite{han2025dialnav} provides a framework for holisti…

Dialog Acts for Task-Driven Embodied Agents

2022-09-26 · Spandana Gella, Aishwarya Padmakumar, Patrick Lange, Dilek Hakkani-Tur

Embodied agents need to be able to interact in natural language understanding task descriptions and asking appropriate follow up questions to obtain necessary information to be effective at successfully accomplishing tas…

Natural Language Understanding

Dialog Acts for Task Driven Embodied Agents

2022-09-01 · SIGDIAL (ACL) 2022 9 · Spandana Gella, Aishwarya Padmakumar, Patrick Lange, Dilek Hakkani-Tur

Embodied agents need to be able to interact in natural language – understanding task descriptions and asking appropriate follow up questions to obtain necessary information to be effective at successfully accomplishing t…

Natural Language Understanding