CigTime: Corrective Instruction Generation Through Inverse Motion Editing
Recent advancements in models linking natural language with human motions have shown significant promise in motion generation and editing based on instructional text. Motivated by applications in sports coaching and motor skill learning, we investigate the inverse problem: generating corrective instructional text, leveraging motion editing and generation models. We introduce a novel approach that, given a user's current motion (source) and the desired motion (target), generates text instructions to guide the user towards achieving the target motion. We leverage large language models to generate corrective texts and utilize existing motion generation and editing frameworks to compile datasets of triplets (source motion, target motion, and corrective text). Using this data, we propose a new motion-language model for generating corrective instructions. We present both qualitative and quantitative results across a diverse range of applications that largely improve upon baselines. Our approach demonstrates its effectiveness in instructional scenarios, offering text-based guidance to correct and enhance user performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingMotion GenerationSimilar Papers 제목 키워드 기반
From Routes to Steps: Separating Semantic Progress from Local Execution in Vision-and-Language Navigation
Vision-and-Language Navigation (VLN) requires an agent to follow a route-level instruction by executing its constituent steps from egocentric visual observations. Existing VLM-based navigators typically supervise both ca…
Pointing to a Llama and Call it a Camel: On the Sycophancy of Multimodal Large Language Models
Multimodal large language models (MLLMs) have demonstrated extraordinary capabilities in conducting conversations based on image inputs. However, we observe that MLLMs exhibit a pronounced form of visual sycophantic beha…
GLLM: Self-Corrective G-Code Generation using Large Language Models with User Feedback
This paper introduces GLLM, an innovative tool that leverages Large Language Models (LLMs) to automatically generate G-code from natural language instructions for Computer Numerical Control (CNC) machining. GLLM addresse…
Code GenerationRAGRetrieval-augmented GenerationNavigational Instruction Generation as Inverse Reinforcement Learning with Neural Machine Translation
Modern robotics applications that involve human-robot interaction require robots to be able to communicate with humans seamlessly and effectively. Natural language provides a flexible and efficient medium through which r…
Machine TranslationNavigatereinforcement-learningReinforcement Learning+2Self-Corrective Task Planning by Inverse Prompting with Large Language Models
In robot task planning, large language models (LLMs) have shown significant promise in generating complex and long-horizon action sequences. However, it is observed that LLMs often produce responses that sound plausible …
Robot Task PlanningTask Planning