Catch & Carry: Reusable Neural Controllers for Vision-Guided Whole-Body Tasks
We address the longstanding challenge of producing flexible, realistic humanoid character controllers that can perform diverse whole-body tasks involving object interactions. This challenge is central to a variety of fields, from graphics and animation to robotics and motor neuroscience. Our physics-based environment uses realistic actuation and first-person perception -- including touch sensors and egocentric vision -- with a view to producing active-sensing behaviors (e.g. gaze direction), transferability to real robots, and comparisons to the biology. We develop an integrated neural-network based approach consisting of a motor primitive module, human demonstrations, and an instructed reinforcement learning regime with curricula and task variations. We demonstrate the utility of our approach for several tasks, including goal-conditioned box carrying and ball catching, and we characterize its behavioral robustness. The resulting controllers can be deployed in real-time on a standard PC. See overview video, https://youtu.be/2rQAW-8gQQk .
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
CleverCatch: A Knowledge-Guided Weak Supervision Model for Fraud Detection
Healthcare fraud detection remains a critical challenge due to limited availability of labeled data, constantly evolving fraud tactics, and the high dimensionality of medical records. Traditional supervised methods are c…
Anomaly DetectionFraud DetectionMemSlides: A Hierarchical Memory Driven Agent Framework for Personalized Slide Generation with Multi-turn Local Revision
Personalized presentation generation requires more than conditioning on a current prompt or template: agents must preserve stable user preferences across tasks, retain newly introduced preferences and constraints during …
CoMic: Co-Training and Mimicry for Reusable Skills
Learning to control complex bodies and reuse learned behaviors is a longstanding challenge in continuous control. We study the problem of learning reusable humanoid skills by imitating motion capture data and co-training…
continuous-controlContinuous ControlReinforcement Learning (RL)Intercepting an Agile Target with Net-Carrying Drones using Competitive Multi-Agent Reinforcement Learning
This article presents a solution to intercept an agile drone by a team of agile drone carrying catching nets. We formulate the problem as a competitive Multi-Agent Reinforcement Learning (MARL) task. To address the probl…
Multi-agent Reinforcement LearningExp2VLA: Enabling Vision-Language-Action for Drone Navigation from Expert Demonstrations
Vision-language-action (VLA) models open a new path toward intuitive robot control by directly linking perception, language, and action in a single end-to-end framework. Yet for UAVs, practical adoption remains difficult…
Reinforcement LearningDrone navigation