"How to best say it?" : Translating Directives in Machine Language into Natural Language in the Blocks World
We propose a method to generate optimal natural language for block placement directives generated by a machine's planner during human-agent interactions in the blocks world. A non user-friendly machine directive, e.g., move(ObjId, toPos), is transformed into visually and contextually grounded referring expressions that are much easier for the user to comprehend. We describe an algorithm that progressively and generatively transforms the machine's directive in ECI (Elementary Composable Ideas)-space, generating many alternative versions of the directive. We then define a cost function to evaluate the ease of comprehension of these alternatives and select the best option. The parameters for this cost function were derived empirically from a user study that measured utterance-to-action timings.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Visually-Grounded Planning without Vision: Language Models Infer Detailed Plans from High-level Instructions
The recently proposed ALFRED challenge task aims for a virtual robotic agent to complete complex multi-step everyday tasks in a virtual home environment from high-level natural language directives, such as {``}put a hot …
TranslationVisually-Grounded Planning without Vision: Language Models Infer Detailed Plans from High-level Instructions
The recently proposed ALFRED challenge task aims for a virtual robotic agent to complete complex multi-step everyday tasks in a virtual home environment from high-level natural language directives, such as "put a hot pie…
TranslationDynamic Code Orchestration: Harnessing the Power of Large Language Models for Adaptive Script Execution
Computer programming initially required humans to directly translate their goals into machine code. These goals could have easily been expressed as a written (or human) language directive. Computers, however, had no capa…
Language ModelingLanguage ModellingLarge Language ModelBeyond the Prompt: An Empirical Study of Cursor Rules
While Large Language Models (LLMs) have demonstrated remarkable capabilities, research shows that their effectiveness depends not only on explicit prompts but also on the broader context provided. This requirement is esp…
Automatic Bilingual Phrase Dictionary Construction from GIZA++ Output
Modern encoder-decoder based neural machine translation (NMT) models are normally trained on parallel sentences. Hence, they give best results when translating full sentences rather than sentence parts. Thereby, the task…
DecoderMachine TranslationNMTSentence+1