Conformal Policy Learning for Sensorimotor Control Under Distribution Shifts
This paper focuses on the problem of detecting and reacting to changes in the distribution of a sensorimotor controller's observables. The key idea is the design of switching policies that can take conformal quantiles as input, which we define as conformal policy learning, that allows robots to detect distribution shifts with formal statistical guarantees. We show how to design such policies by using conformal quantiles to switch between base policies with different characteristics, e.g. safety or speed, or directly augmenting a policy observation with a quantile and training it with reinforcement learning. Theoretically, we show that such policies achieve the formal convergence guarantees in finite time. In addition, we thoroughly evaluate their advantages and limitations on two compelling use cases: simulated autonomous driving and active perception with a physical quadruped. Empirical results demonstrate that our approach outperforms five baselines. It is also the simplest of the baseline strategies besides one ablation. Being easy to use, flexible, and with formal guarantees, our work demonstrates how conformal prediction can be an effective tool for sensorimotor learning under uncertainty.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous DrivingConformal PredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Robust Conformal CBF and CLF Controllers via Iterative Policy Updates
Conformal prediction (CP) has been used to obtain probabilistic bounds on the error between a learned dynamics model and the true but unknown system. Such CP bounds can then be embedded into robust control Lyapunov funct…
Active Vision Reinforcement Learning under Limited Visual Observability
In this work, we investigate Active Vision Reinforcement Learning (ActiveVision-RL), where an embodied agent simultaneously learns action policy for the task while also controlling its visual observations in partially ob…
reinforcement-learningReinforcement LearningConformal Policy Learning with Distribution-Free Safety Guarantees
Policy learning aims to determine who should be treated based on individual characteristics. In high-stakes settings such as medicine and public policy where safety is a central concern, improving the average outcomes al…
Conformal Off-Policy Evaluation in Markov Decision Processes
Reinforcement Learning aims at identifying and evaluating efficient control policies from data. In many real-world applications, the learner is not allowed to experiment and cannot gather data in an online manner (this i…
Conformal PredictionOff-policy evaluationNeural Dynamic Policies for End-to-End Sensorimotor Learning
The current dominant paradigm in sensorimotor control, whether imitation or reinforcement learning, is to train policies directly in raw action spaces such as torque, joint angle, or end-effector position. This forces th…
Imitation Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)