paper-with-me

Papers

Solving Dialogue Grounding Embodied Task in a Simulated Environment using Further Masked Language Modeling

2023-06-21 · Weijie Jack Zhang

Enhancing AI systems with efficient communication skills that align with human understanding is crucial for their effective assistance to human users. Proactive initiatives from the system side are needed to discern specific circumstances and interact aptly with users to solve these scenarios. In this research, we opt for a collective building assignment taken from the Minecraft dataset. Our proposed method employs language modeling to enhance task understanding through state-of-the-art (SOTA) methods using language models. These models focus on grounding multi-modal understandinging and task-oriented dialogue comprehension tasks. This focus aids in gaining insights into how well these models interpret and respond to a variety of inputs and tasks. Our experimental results provide compelling evidence of the superiority of our proposed method. This showcases a substantial improvement and points towards a promising direction for future research in this domain.

📄 PDF Abstract BibTeX arXiv:2306.12387

Code (0)

등록된 구현이 없습니다.

Tasks

Language ModelingLanguage ModellingMasked Language ModelingMinecraft

Methods 이 논문이 사용한 방법론

OPT OPT is a suite of decoder-only pre-trained transformers ranging from 125M to 175B parameters. The model uses an AdamW optimizer and weight decay of 0.1. It follows a linear…
ALIGN In the ALIGN method, visual and language representations are jointly trained from noisy image alt-text data. The image and text encoders are learned via contrastive loss…
Focus 설명 없음

Similar Papers 제목 키워드 기반

Demonstrating EMMA: Embodied MultiModal Agent for Language-guided Action Execution in 3D Simulated Environments

2022-09-01 · SIGDIAL (ACL) 2022 9 · Alessandro Suglia, Bhathiya Hemanthage, Malvina Nikandrou, George Pantazopoulos 외

We demonstrate EMMA, an embodied multimodal agent which has been developed for the Alexa Prize SimBot challenge. The agent acts within a 3D simulated environment for household tasks. EMMA is a unified and multimodal gene…

Conditional Text GenerationText Generation

RoboBlockly Studio: Conversational Block Programming with Embodied Robot Feedback for Computational Thinking

2026-05-12 · Leyi Li, Chenyu Du, Jiafei Sun, Erick Purwanto 외 arxiv

Computational thinking (CT) is increasingly promoted as a core literacy, yet learners and teachers face challenges in connecting abstract program logic to meaningful outcomes. We design and evaluate RoboBlockly Studio, a…

SECURE: Semantics-aware Embodied Conversation under Unawareness for Lifelong Robot Learning

2024-09-26 · Rimvydas Rubavicius, Peter David Fagan, Alex Lascarides, Subramanian Ramamoorthy

This paper addresses a challenging interactive task learning scenario we call rearrangement under unawareness: to manipulate a rigid-body environment in a context where the agent is unaware of a concept that is key to so…

Novel ConceptsSentence

TEACh: Task-driven Embodied Agents that Chat

2021-10-01 · Aishwarya Padmakumar, Jesse Thomason, Ayush Shrivastava, Patrick Lange 외

Robots operating in human spaces must be able to engage in natural language interaction with people, both understanding and executing instructions, and using conversation to resolve ambiguity and recover from mistakes. T…

Dialogue Understanding

Semantic Grounding in Dialogue for Complex Problem Solving

2015-05-01 · HLT 2015 5 · Xiaolong Li, Kristy Boyer
Natural Language Understanding