Structured Control Nets for Deep Reinforcement Learning

2018-02-22 · ICML 2018 7 · Mario Srouji, Jian Zhang, Ruslan Salakhutdinov

In recent years, Deep Reinforcement Learning has made impressive advances in solving several important benchmark problems for sequential decision making. Many control applications use a generic multilayer perceptron (MLP) for non-vision parts of the policy network. In this work, we propose a new neural network architecture for the policy network representation that is simple yet effective. The proposed Structured Control Net (SCN) splits the generic MLP into two separate sub-modules: a nonlinear control module and a linear control module. Intuitively, the nonlinear control is for forward-looking and global control, while the linear control stabilizes the local dynamics around the residual of global control. We hypothesize that this will bring together the benefits of both linear and nonlinear policies: improve training sample efficiency, final episodic reward, and generalization of learned policy, while requiring a smaller network and being generally applicable to different training methods. We validated our hypothesis with competitive results on simulations from OpenAI MuJoCo, Roboschool, Atari, and a custom 2D urban driving environment, with various ablation and generalization tests, trained with multiple black-box and policy gradient training methods. The proposed architecture has the potential to improve upon broader control tasks by incorporating problem specific priors into the architecture. As a case study, we demonstrate much improved performance for locomotion tasks by emulating the biological central pattern generators (CPGs) as the nonlinear part of the architecture.

📄 PDF Abstract BibTeX arXiv:1802.08311

Code (1)

wongongv/scnwithdqn tf

Tasks

Decision MakingDeep Reinforcement LearningMuJoCoreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Sequential Decision Making

CFlowNets: Continuous Control with Generative Flow Networks

2023-03-04 · Yinchuan Li, Shuang Luo, Haozhi Wang, Jianye Hao

Generative flow networks (GFlowNets), as an emerging technique, can be used as an alternative to reinforcement learning for exploratory control tasks. GFlowNet aims to generate distribution proportional to the rewards ov…

Active Learning continuous-control Continuous Control reinforcement-learning+2

Recurrent Control Nets for Deep Reinforcement Learning

2019-01-06 · Vincent Liu, Ademi Adeniji, Nathaniel Lee, Jason Zhao 외

Central Pattern Generators (CPGs) are biological neural circuits capable of producing coordinated rhythmic outputs in the absence of rhythmic input. As a result, they are responsible for most rhythmic motion in living or…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Comparing Behavioural Cloning and Reinforcement Learning for Spacecraft Guidance and Control Networks

2025-07-22 · Harry Holt, Sebastien Origer, Dario Izzo arxiv

Guidance & control networks (G&CNETs) provide a promising alternative to on-board guidance and control (G&C) architectures for spacecraft, offering a differentiable, end-to-end representation of the guidance and control …

Reinforcement Learning

ReasoNet: Learning to Stop Reading in Machine Comprehension

2016-09-17 · Yelong Shen, Po-Sen Huang, Jianfeng Gao, Weizhu Chen

Teaching a computer to read and answer general questions pertaining to a document is a challenging yet unsolved problem. In this paper, we describe a novel neural network architecture called the Reasoning Network (ReasoN…

Question Answering Reading Comprehension Reinforcement Learning

SE3-Pose-Nets: Structured Deep Dynamics Models for Visuomotor Planning and Control

2017-10-02 · Arunkumar Byravan, Felix Leeb, Franziska Meier, Dieter Fox

In this work, we present an approach to deep visuomotor control using structured deep dynamics models. Our deep dynamics model, a variant of SE3-Nets, learns a low-dimensional pose embedding for visuomotor control via an…

Decoder