CAPACITY-LIMITED REINFORCEMENT LEARNING: APPLICATIONS IN DEEP ACTOR-CRITIC METHODS FOR CONTINUOUS CONTROL

2019-09-25 · Tyler James Malloy, Matthew Riemer, Miao Liu, Tim Klinger, Gerald Tesauro, Chris R. Sims

Biological and artificial agents must learn to act optimally in spite of a limited capacity for processing, storing, and attending to information. We formalize this type of bounded rationality in terms of an information-theoretic constraint on the complexity of policies that agents seek to learn. We present the Capacity-Limited Reinforcement Learning (CLRL) objective which defines an optimal policy subject to an information capacity constraint. This objective is optimized by drawing from methods used in rate distortion theory and information theory, and applied to the reinforcement learning setting. Using this objective we implement a novel Capacity-Limited Actor-Critic (CLAC) algorithm and situate it within a broader family of RL algorithms such as the Soft Actor Critic (SAC) and discuss their similarities and differences. Our experiments show that compared to alternative approaches, CLAC offers improvements in generalization between training and modified test environments. This is achieved in the CLAC model while displaying high sample efficiency and minimal requirements for hyper-parameter tuning.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

continuous-control Continuous Control reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Policy Generalization In Capacity-Limited Reinforcement Learning

2018-09-27 · Rachel A. Lerch, Chris R. Sims

Motivated by the study of generalization in biological intelligence, we examine reinforcement learning (RL) in settings where there are information-theoretic constraints placed on the learner's ability to represent a beh…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Deep RL With Information Constrained Policies: Generalization in Continuous Control

2020-10-09 · Tyler Malloy, Chris R. Sims, Tim Klinger, Miao Liu 외

Biological agents learn and act intelligently in spite of a highly limited capacity to process and store information. Many real-world problems involve continuous control, which represents a difficult task for artificial …

continuous-control Continuous Control reinforcement-learning Reinforcement Learning+1

Simple Actors and Deep Critics for Scalable Reinforcement Learning

2026-08-27 · Guhyeon Kang, Jaehwi Lee, Minhae Kwon arxiv

Recent progress in offline reinforcement learning (RL) has been driven by expressive generative actors such as diffusion and flow-matching policies, which capture multimodal behavior in offline datasets. However, these a…

Reinforcement Learning Offline RL

Optimization Solution Functions as Deterministic Policies for Offline Reinforcement Learning

2024-08-27 · Vanshaj Khattar, Ming Jin

Offline reinforcement learning (RL) is a promising approach for many control applications but faces challenges such as limited data coverage and value function overestimation. In this paper, we propose an implicit actor-…

Offline RL reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Finite-Time Global Optimality Convergence in Deep Neural Actor-Critic Methods for Decentralized Multi-Agent Reinforcement Learning

2025-05-24 · Zhiyao Zhang, Myeung Suk Oh, FNU Hairi, Ziyue Luo 외

Actor-critic methods for decentralized multi-agent reinforcement learning (MARL) facilitate collaborative optimal decision making without centralized coordination, thus enabling a wide range of applications in practice. …

Decision Making Multi-agent Reinforcement Learning