Combining Reinforcement Learning and Tensor Networks, with an Application to Dynamical Large Deviations

2022-09-28 · Edward Gillman, Dominic C. Rose, Juan P. Garrahan

We present a framework to integrate tensor network (TN) methods with reinforcement learning (RL) for solving dynamical optimisation tasks. We consider the RL actor-critic method, a model-free approach for solving RL problems, and introduce TNs as the approximators for its policy and value functions. Our "actor-critic with tensor networks" (ACTeN) method is especially well suited to problems with large and factorisable state and action spaces. As an illustration of the applicability of ACTeN we solve the exponentially hard task of sampling rare trajectories in two paradigmatic stochastic models, the East model of glasses and the asymmetric simple exclusion process (ASEP), the latter being particularly challenging to other methods due to the absence of detailed balance. With substantial potential for further integration with the vast array of existing RL methods, the approach introduced here is promising both for applications in physics and to multi-agent RL problems more generally.

📄 PDF Abstract BibTeX arXiv:2209.14089

Code (1)

rl-with-tns/acten_code 공식 구현 jax

Tasks

reinforcement-learning Reinforcement Learning (RL)Tensor Networks

LIFT: Reinforcement Learning in Computer Systems by Learning From Demonstrations

2018-08-23 · Michael Schaarschmidt, Alexander Kuhnle, Ben Ellis, Kai Fricke 외

Reinforcement learning approaches have long appealed to the data management community due to their ability to learn to control dynamic behavior from raw system performance. Recent successes in combining deep neural netwo…

Deep Reinforcement Learning Management reinforcement-learning Reinforcement Learning+1

A Tensor Network Approach to Finite Markov Decision Processes

2020-02-12 · Edward Gillman, Dominic C. Rose, Juan P. Garrahan

Tensor network (TN) techniques - often used in the context of quantum many-body physics - have shown promise as a tool for tackling machine learning (ML) problems. The application of TNs to ML, however, has mostly focuse…

Reinforcement Learning Reinforcement Learning (RL)

Reinforcement Learning for Optimizing Large Qubit Array based Quantum Sensor Circuits

2025-08-28 · Laxmisha Ashok Attisara, Sathish Kumar arxiv

As the number of qubits in a sensor increases, the complexity of designing and controlling the quantum circuits grows exponentially. Manually optimizing these circuits becomes infeasible. Optimizing entanglement distribu…

Quantum Machine Learning Reinforcement Learning

Tensor networks for unsupervised machine learning

2021-06-24 · Jing Liu, Sujie Li, Jiang Zhang, Pan Zhang

Modeling the joint distribution of high-dimensional data is a central task in unsupervised machine learning. In recent years, many interests have been attracted to developing learning models based on tensor networks, whi…

BIG-bench Machine Learning Tensor Networks

Hybrid Reinforcement Learning with Expert State Sequences

2019-03-11 · Xiaoxiao Guo, Shiyu Chang, Mo Yu, Gerald Tesauro 외

Existing imitation learning approaches often require that the complete demonstration data, including sequences of actions and states, are available. In this paper, we consider a more realistic and difficult scenario wher…

Atari Games Imitation Learning reinforcement-learning Reinforcement Learning+1