Retrieval-Augmented Code Generation for Situated Action Generation: A Case Study on Minecraft
In the Minecraft Collaborative Building Task, two players collaborate: an Architect (A) provides instructions to a Builder (B) to assemble a specified structure using 3D blocks. In this work, we investigate the use of large language models (LLMs) to predict the sequence of actions taken by the Builder. Leveraging LLMs' in-context learning abilities, we use few-shot prompting techniques, that significantly improve performance over baseline methods. Additionally, we present a detailed analysis of the gaps in performance for future work
Code (0)
등록된 구현이 없습니다.
Tasks
Action GenerationCode GenerationIn-Context LearningMinecraftRetrievalSimilar Papers 제목 키워드 기반
SitEmb-v1.5: Improved Context-Aware Dense Retrieval for Semantic Association and Long Story Comprehension
Retrieval-augmented generation (RAG) over long documents typically involves splitting the text into smaller chunks, which serve as the basic units for retrieval. However, due to dependencies across the original document,…
Mix-of-Granularity: Optimize the Chunking Granularity for Retrieval-Augmented Generation
Integrating information from various reference databases is a major challenge for Retrieval-Augmented Generation (RAG) systems because each knowledge source adopts a unique data structure and follows different convention…
ChunkingRAGRetrievalRetrieval-augmented GenerationLearning to Embed Multi-Modal Contexts for Situated Conversational Agents
The Situated Interactive Multi-Modal Conversations (SIMMC) 2.0 aims to create virtual shopping assistants that can accept complex multi-modal inputs, i.e. visual appearances of objects and user utterances. It consists of…
coreference-resolutionCoreference ResolutionDecoderdialog state tracking+2Learning to Embed Multi-Modal Contexts for Situated Conversational Agents
The Situated Interactive Multi-Modal Conversations (SIMMC) 2.0 aims to create virtual shopping assistants that can accept complex multi-modal inputs, i.e. visual appearances of objects and user utterances. It consists of…
coreference-resolutionCoreference ResolutionDecoderdialog state tracking+3ARCS: Agentic Retrieval-Augmented Code Synthesis with Iterative Refinement
In supercomputing, efficient and optimized code generation is essential to leverage high-performance systems effectively. We propose Agentic Retrieval-Augmented Code Synthesis (ARCS), an advanced framework for accurate, …
Code GenerationHumanEvalRAGRetrieval+2