Policy Gradient RL Algorithms as Directed Acyclic Graphs

2020-12-14 · Juan Jose Garau Luis

Meta Reinforcement Learning (RL) methods focus on automating the design of RL algorithms that generalize to a wide range of environments. The framework introduced in (Anonymous, 2020) addresses the problem by representing different RL algorithms as Directed Acyclic Graphs (DAGs), and using an evolutionary meta learner to modify these graphs and find good agent update rules. While the search language used to generate graphs in the paper serves to represent numerous already-existing RL algorithms (e.g., DQN, DDQN), it has limitations when it comes to representing Policy Gradient algorithms. In this work we try to close this gap by extending the original search language and proposing graphs for five different Policy Gradient algorithms: VPG, PPO, DDPG, TD3, and SAC.

📄 PDF Abstract BibTeX arXiv:2012.07763

Code (1)

jjgarau/DAGPolicyGradient 공식 구현

Tasks

Meta Reinforcement Learningreinforcement-learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Dilated Convolution 설명 없음

Global Average Pooling Global Average Pooling is a pooling operation designed to replace fully connected layers in classical CNNs. The idea is to generate one feature map for each corresponding…

Average Pooling 설명 없음

1x1 Convolution A 1 x 1 Convolution is a convolution with some special properties in that it can be used for dimensionality reduction,…

SAC 설명 없음

Batch Normalization 설명 없음

Adam 설명 없음

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

Supplement to "Reversible MCMC on Markov equivalence classes of sparse directed acyclic graphs"

2013-03-04 · Yangbo He, Jinzhu Jia, Bin Yu

This supplementary material includes three parts: some preliminary results, four examples, an experiment, three new algorithms, and all proofs of the results in the paper "Reversible MCMC on Markov equivalence classes of…

Multi-Task Semantic Dependency Parsing with Policy Gradient for Learning Easy-First Strategies

2019-06-04 · ACL 2019 7 · Shuhei Kurita, Anders Søgaard

In Semantic Dependency Parsing (SDP), semantic relations form directed acyclic graphs, rather than trees. We propose a new iterative predicate selection (IPS) algorithm for SDP. Our IPS algorithm combines the graph-based…

Dependency Parsing Multi-Task Learning Semantic Dependency Parsing

Multilevel Acyclic Hypergraph Partitioning

2020-02-06 · Merten Popp, Sebastian Schlag, Christian Schulz, Daniel Seemaier

A directed acyclic hypergraph is a generalized concept of a directed acyclic graph, where each hyperedge can contain an arbitrary number of tails and heads. Directed hypergraphs can be used to model data flow and executi…

hypergraph partitioning Scheduling

Lower Bounds on the Size of Markov Equivalence Classes

2025-06-26 · Erik Jahn, Frederick Eberhardt, Leonard J. Schulman

Causal discovery algorithms typically recover causal graphs only up to their Markov equivalence classes unless additional parametric assumptions are made. The sizes of these equivalence classes reflect the limits of what…

Causal Discovery

Spectral clustering algorithms for the detection of clusters in block-cyclic and block-acyclic graphs

2018-05-02 · H. Van Lierde, T. W. S. Chow, J. -C. Delvenne

We propose two spectral algorithms for partitioning nodes in directed graphs respectively with a cyclic and an acyclic pattern of connection between groups of nodes. Our methods are based on the computation of extremal e…

Clustering Graph Clustering