Adaptive Batch Size for Safe Policy Gradients
Policy gradient methods are among the best Reinforcement Learning (RL) techniques to solve complex control problems. In real-world RL applications, it is common to have a good initial policy whose performance needs to be improved and it may not be acceptable to try bad policies during the learning process. Although several methods for choosing the step size exist, research paid less attention to determine the batch size, that is the number of samples used to estimate the gradient direction for each update of the policy parameters. In this paper, we propose a set of methods to jointly optimize the step and the batch sizes that guarantee (with high probability) to improve the policy performance after each update. Besides providing theoretical guarantees, we show numerical simulations to analyse the behaviour of our methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Policy Gradient MethodsReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Smoothing Policies and Safe Policy Gradients
Policy Gradient (PG) algorithms are among the best candidates for the much-anticipated applications of reinforcement learning to real-world control tasks, such as robotics. However, the trial-and-error nature of these me…
Reinforcement LearningStochastic OptimizationKFCPO: Kronecker-Factored Approximated Constrained Policy Optimization
We propose KFCPO, a novel Safe Reinforcement Learning (Safe RL) algorithm that combines scalable Kronecker-Factored Approximate Curvature (K-FAC) based second-order policy optimization with safety-aware gradient manipula…
Reinforcement LearningBig Batch SGD: Automated Inference using Adaptive Batch Sizes
Classical stochastic gradient methods for optimization rely on noisy gradient approximations that become progressively less accurate as iterates approach a solution. The large noise and small signal in the resulting grad…
See through Gradients: Image Batch Recovery via GradInversion
Training deep neural networks requires gradient estimation from data batches to update parameters. Gradients per parameter are averaged over a set of data and this has been presumed to be safe for privacy-preserving trai…
Federated LearningInference AttackPrivacy PreservingCommunication-Efficient Adaptive Batch Size Strategies for Distributed Local Gradient Methods
Modern deep neural networks often require distributed training with many workers due to their large size. As the number of workers increases, communication overheads become the main bottleneck in data-parallel minibatch …
image-classificationImage ClassificationLanguage ModelingLanguage Modelling