Solving Dialogue Grounding Embodied Task in a Simulated Environment using Further Masked Language Modeling
Enhancing AI systems with efficient communication skills that align with human understanding is crucial for their effective assistance to human users. Proactive initiatives from the system side are needed to discern specific circumstances and interact aptly with users to solve these scenarios. In this research, we opt for a collective building assignment taken from the Minecraft dataset. Our proposed method employs language modeling to enhance task understanding through state-of-the-art (SOTA) methods using language models. These models focus on grounding multi-modal understandinging and task-oriented dialogue comprehension tasks. This focus aids in gaining insights into how well these models interpret and respond to a variety of inputs and tasks. Our experimental results provide compelling evidence of the superiority of our proposed method. This showcases a substantial improvement and points towards a promising direction for future research in this domain.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingMasked Language ModelingMinecraftMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Demonstrating EMMA: Embodied MultiModal Agent for Language-guided Action Execution in 3D Simulated Environments
We demonstrate EMMA, an embodied multimodal agent which has been developed for the Alexa Prize SimBot challenge. The agent acts within a 3D simulated environment for household tasks. EMMA is a unified and multimodal gene…
Conditional Text GenerationText GenerationRoboBlockly Studio: Conversational Block Programming with Embodied Robot Feedback for Computational Thinking
Computational thinking (CT) is increasingly promoted as a core literacy, yet learners and teachers face challenges in connecting abstract program logic to meaningful outcomes. We design and evaluate RoboBlockly Studio, a…
SECURE: Semantics-aware Embodied Conversation under Unawareness for Lifelong Robot Learning
This paper addresses a challenging interactive task learning scenario we call rearrangement under unawareness: to manipulate a rigid-body environment in a context where the agent is unaware of a concept that is key to so…
Novel ConceptsSentenceTEACh: Task-driven Embodied Agents that Chat
Robots operating in human spaces must be able to engage in natural language interaction with people, both understanding and executing instructions, and using conversation to resolve ambiguity and recover from mistakes. T…
Dialogue Understanding