Rich State Observations Empower Reinforcement Learning to Surpass PID: A Drone Ball Balancing Study

2025-09-25 · Mingjiang Liu, Hailong Huang arxiv

This paper addresses a drone ball-balancing task, in which a drone stabilizes a ball atop a movable beam through cable-based interaction. We propose a hierarchical control framework that decouples high-level balancing policy from low-level drone control, and train a reinforcement learning (RL) policy to handle the high-level decision-making. Simulation results show that the RL policy achieves superior performance compared to carefully tuned PID controllers within the same hierarchical structure. Through systematic comparative analysis, we demonstrate that RL's advantage stems not from improved parameter tuning or inherent nonlinear mapping capabilities, but from its ability to effectively utilize richer state observations. These findings underscore the critical role of comprehensive state representation in learning-based systems and suggest that enhanced sensing could be instrumental in improving controller performance.

📄 PDF Abstract BibTeX arXiv:2509.21122

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

Can Temporal-Diﬀerence and Q-Learning Learn Representation? A Mean-Field Theory

2020-12-01 · NeurIPS 2020 12 · Yufeng Zhang, Qi Cai, Zhuoran Yang, Yongxin Chen 외

Temporal-diﬀerence and Q-learning play a key role in deep reinforcement learning, where they are empowered by expressive nonlinear function approximators such as neural networks. At the core of their empirical successes …

Deep Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning+1

ReTouch: Empowering Contact-Rich Dexterous Manipulation with Online-Refined Tactile Prediction

2026-08-03 · Shiqi Zhang, Xin Zhang, Yedong Shen, Yao Li 외 arxiv

Fusing tactile signals has proven effective for contact-rich manipulation, enabling robots to perceive contact states and adapt to rapidly changing physical interactions. Yet effectively integrating tactile feedback into…

On Oracle-Efficient PAC RL with Rich Observations

2018-03-01 · NeurIPS 2018 12 · Christoph Dann, Nan Jiang, Akshay Krishnamurthy, Alekh Agarwal 외

We study the computational tractability of PAC reinforcement learning with rich observations. We present new provably sample-efficient algorithms for environments with deterministic hidden state dynamics and stochastic r…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Learning to Perceive the World Through Control: Empowerment-Based Representation Learning

2026-05-28 · Mahsa Bastankhah, Sophie Broderick, Benjamin Eysenbach arxiv

In many practical reinforcement learning environments, observations are far higher-dimensional than the variables that matter for control. In this work, we ask: can we learn representations that capture only control-rele…

Representation Learning Reinforcement Learning

Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning

2026-02-10 · Zhaoyang Wang, Canwen Xu, Boyi Liu, Yite Wang 외 arxiv

Recent advances in large language model (LLM) have empowered autonomous agents to perform multi-turn interactions with tools and environments. However, scaling such agent training is limited by the lack of diverse and re…