Parameter Estimation using Reinforcement Learning Causal Curiosity: Limits and Challenges

2025-05-13 · Miguel Arana-Catania, Weisi Guo

Causal understanding is important in many disciplines of science and engineering, where we seek to understand how different factors in the system causally affect an experiment or situation and pave a pathway towards creating effective or optimising existing models. Examples of use cases are autonomous exploration and modelling of unknown environments or assessing key variables in optimising large complex systems. In this paper, we analyse a Reinforcement Learning approach called Causal Curiosity, which aims to estimate as accurately and efficiently as possible, without directly measuring them, the value of factors that causally determine the dynamics of a system. Whilst the idea presents a pathway forward, measurement accuracy is the foundation of methodology effectiveness. Focusing on the current causal curiosity's robotic manipulator, we present for the first time a measurement accuracy analysis of the future potentials and current limitations of this technique and an analysis of its sensitivity and confounding factor disentanglement capability - crucial for causal analysis. As a result of our work, we promote proposals for an improved and efficient design of Causal Curiosity methods to be applied to real-world complex scenarios.

📄 PDF Abstract BibTeX arXiv:2505.08453

Code (0)

등록된 구현이 없습니다.

Tasks

Disentanglementparameter estimation

Causal Counterfactuals for Improving the Robustness of Reinforcement Learning

2022-11-02 · Tom He, Jasmina Gajcin, Ivana Dusparic

Reinforcement learning (RL) is used in various robotic applications. RL enables agents to learn tasks autonomously by interacting with the environment. The more critical the tasks are, the higher the demand for the robus…

Causal Inference reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Hyper: Hyperparameter Robust Efficient Exploration in Reinforcement Learning

2024-12-04 · Yiran Wang, Chenshu Liu, Yunfan Li, Sanae Amani 외

The exploration \& exploitation dilemma poses significant challenges in reinforcement learning (RL). Recently, curiosity-based exploration methods achieved great success in tackling hard-exploration problems. However, th…

Efficient Exploration reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learning

2020-10-07 · Sumedh A. Sontakke, Arash Mehrjou, Laurent Itti, Bernhard Schölkopf

Animals exhibit an innate ability to learn regularities of the world through interaction. By performing experiments in their environment, they are able to discern the causal factors of variation and infer how they affect…

Representation Learning Zero-Shot Learning

Wind Estimation in Unmanned Aerial Vehicles with Causal Machine Learning

2024-07-01 · Abdulaziz Alwalan, Miguel Arana-Catania

In this work we demonstrate the possibility of estimating the wind environment of a UAV without specialised sensors, using only the UAV's trajectory, applying a causal machine learning approach. We implement the causal c…

Clustering

Curiosity Based Reinforcement Learning on Robot Manufacturing Cell

2020-11-17 · Mohammed Sharafath Abdul Hameed, Md Muzahid Khan, Andreas Schwung

This paper introduces a novel combination of scheduling control on a flexible robot manufacturing cell with curiosity based reinforcement learning. Reinforcement learning has proved to be highly successful in solving tas…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Scheduling