Demonstration-Free Robotic Control via LLM Agents

2026-01-28 · Brian Y. Tsui, Alan Y. Fang, Tiffany J. Hwu arxiv

Robotic manipulation has increasingly adopted vision-language-action (VLA) models, which achieve strong performance but typically require task-specific demonstrations and fine-tuning, and often generalize poorly under domain shift. We investigate whether general-purpose large language model (LLM) agent frameworks, originally developed for software engineering, can serve as an alternative control paradigm for embodied manipulation. We introduce FAEA (Frontier Agent as Embodied Agent), which applies an LLM agent framework directly to embodied manipulation without modification. Using the same iterative reasoning that enables software agents to debug code, FAEA enables embodied agents to reason through manipulation strategies. We evaluate an unmodified frontier agent, Claude Agent SDK, across the LIBERO, ManiSkill3, and MetaWorld benchmarks. With privileged environment state access, FAEA achieves success rates of 84.9%, 85.7%, and 96%, respectively. This level of task success approaches that of VLA models trained with less than 100 demonstrations per task, without requiring demonstrations or fine-tuning. With one round of human feedback as an optional optimization, performance increases to 88.2% on LIBERO. This demonstration-free capability has immediate practical value: FAEA can autonomously explore novel scenarios in simulation and generate successful trajectories for training data augmentation in embodied learning. Our results indicate that general-purpose agents are sufficient for a class of manipulation tasks dominated by deliberative, task-level planning. This opens a path for robotics systems to leverage actively maintained agent infrastructure and benefit directly from ongoing advances in frontier models. Code is available at https://github.com/robiemusketeer/faea-sim

📄 PDF Abstract BibTeX arXiv:2601.20334

Code (0)

등록된 구현이 없습니다.

Tasks

Data Augmentation

Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards

2017-07-27 · Mel Vecerik, Todd Hester, Jonathan Scholz, Fumin Wang 외

We propose a general and model-free approach for Reinforcement Learning (RL) on real robotics with sparse rewards. We build upon the Deep Deterministic Policy Gradient (DDPG) algorithm to use demonstrations. Both demonst…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Dexterous Robotic Piano Playing at Scale

2025-11-04 · Le Chen, Yi Zhao, Jan Schneider, Quankai Gao 외 arxiv

Endowing robot hands with human-level dexterity has been a long-standing goal in robotics. Bimanual robotic piano playing represents a particularly challenging task: it is high-dimensional, contact-rich, and requires fas…

Reinforcement Learning

Model-based actor-critic: GAN (model generator) + DRL (actor-critic) => AGI

2020-04-04 · Aras Dargazany

Our effort is toward unifying GAN and DRL algorithms into a unifying AI model (AGI or general-purpose AI or artificial general intelligence which has general-purpose applications to: (A) offline learning (of stored data)…

model OpenAI Gym Unity

AdaDemo: Data-Efficient Demonstration Expansion for Generalist Robotic Agent

2024-04-11 · Tongzhou Mu, Yijie Guo, Jie Xu, Ankit Goyal 외

Encouraged by the remarkable achievements of language and vision foundation models, developing generalist robotic agents through imitation learning, using large demonstration datasets, has become a prominent area of inte…

Imitation Learning

Lightweight Learning from Actuation-Space Demonstrations via Flow Matching for Whole-Body Soft Robotic Grasping

2025-11-03 · Liudi Yang, Yang Bai, Yuhao Wang, Ibrahim Alsarraj 외 arxiv

Robotic grasping under uncertainty remains a fundamental challenge due to its uncertain and contact-rich nature. Traditional rigid robotic hands, with limited degrees of freedom and compliance, rely on complex model-base…

Robotic Grasping