홈 › Papers

Provably Robust Blackbox Optimization for Reinforcement Learning

2019-03-07 · Krzysztof Choromanski, Aldo Pacchiano, Jack Parker-Holder, Yunhao Tang, Deepali Jain, Yuxiang Yang, Atil Iscen, Jasmine Hsu, Vikas Sindhwani

Interest in derivative-free optimization (DFO) and "evolutionary strategies" (ES) has recently surged in the Reinforcement Learning (RL) community, with growing evidence that they can match state of the art methods for policy optimization problems in Robotics. However, it is well known that DFO methods suffer from prohibitively high sampling complexity. They can also be very sensitive to noisy rewards and stochastic dynamics. In this paper, we propose a new class of algorithms, called Robust Blackbox Optimization (RBO). Remarkably, even if up to $23\%$ of all the measurements are arbitrarily corrupted, RBO can provably recover gradients to high accuracy. RBO relies on learning gradient flows using robust regression methods to enable off-policy updates. On several MuJoCo robot control tasks, when all other RL approaches collapse in the presence of adversarial noise, RBO is able to train policies effectively. We also show that RBO can be applied to legged locomotion tasks including path tracking for quadruped robots.

📄 PDF Abstract BibTeX arXiv:1903.02993

Code (0)

등록된 구현이 없습니다.

Tasks

MuJoCoreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Text-to-Image Generation

From Complexity to Simplicity: Adaptive ES-Active Subspaces for Blackbox Optimization

2019-03-07 · NeurIPS 2019 12 · Krzysztof Choromanski, Aldo Pacchiano, Jack Parker-Holder, Yunhao Tang

We present a new algorithm ASEBO for optimizing high-dimensional blackbox functions. ASEBO adapts to the geometry of the function and learns optimal sets of sensing directions, which are used to probe it, on-the-fly. It …

compressed sensing Multi-Armed Bandits Reinforcement Learning

CoNES: Convex Natural Evolutionary Strategies

2020-07-16 · Sushant Veer, Anirudha Majumdar

We present a novel algorithm -- convex natural evolutionary strategies (CoNES) -- for optimizing high-dimensional blackbox functions by leveraging tools from convex optimization and information geometry. CoNES is formula…

Benchmarking MuJoCo reinforcement-learning Reinforcement Learning (RL)

On Blackbox Backpropagation and Jacobian Sensing

2017-12-01 · NeurIPS 2017 12 · Krzysztof M. Choromanski, Vikas Sindhwani

From a small number of calls to a given “blackbox" on random input perturbations, we show how to efficiently recover its unknown Jacobian, or estimate the left action of its Jacobian on a given vector. Our methods are ba…

compressed sensing

Learning and Optimization of Blackbox Combinatorial Solvers in Neural Networks

2020-06-06 · T. J. Wilder

The use of blackbox solvers inside neural networks is a relatively new area which aims to improve neural network performance by including proven, efficient solvers for complex problems. Existing work has created methods …

Provable Safe Reinforcement Learning with Binary Feedback

2022-10-26 · Andrew Bennett, Dipendra Misra, Nathan Kallus

Safety is a crucial necessity in many applications of reinforcement learning (RL), whether robotic, automotive, or medical. Many existing approaches to safe RL rely on receiving numeric safety feedback, but in many cases…

Active Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1