Differentially Encoded Observation Spaces for Perceptive Reinforcement Learning

2023-10-03 · Lev Grossman, Brian Plancher

Perceptive deep reinforcement learning (DRL) has lead to many recent breakthroughs for complex AI systems leveraging image-based input data. Applications of these results range from super-human level video game agents to dexterous, physically intelligent robots. However, training these perceptive DRL-enabled systems remains incredibly compute and memory intensive, often requiring huge training datasets and large experience replay buffers. This poses a challenge for the next generation of field robots that will need to be able to learn on the edge in order to adapt to their environments. In this paper, we begin to address this issue through differentially encoded observation spaces. By reinterpreting stored image-based observations as a video, we leverage lossless differential video encoding schemes to compress the replay buffer without impacting training performance. We evaluate our approach with three state-of-the-art DRL algorithms and find that differential image encoding reduces the memory footprint by as much as 14.2x and 16.7x across tasks from the Atari 2600 benchmark and the DeepMind Control Suite (DMC) respectively. These savings also enable large-scale perceptive DRL that previously required paging between flash and RAM to be run entirely in RAM, improving the latency of DMC tasks by as much as 32%.

📄 PDF Abstract BibTeX arXiv:2310.01767

Code (1)

a2r-lab/diffcompressdrl 공식 구현

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning

Methods 이 논문이 사용한 방법론

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

Gait-Adaptive Perceptive Humanoid Locomotion with Real-Time Under-Base Terrain Reconstruction

2025-12-08 · Haolin Song, Hongbo Zhu, Tao Yu, Yan Liu 외 arxiv

For full-size humanoid robots, even with recent advances in reinforcement learning-based control, achieving reliable locomotion on complex terrains, such as long staircases, remains challenging. In such settings, limited…

Reinforcement Learning

Learning Perceptive Platform Adaptive Locomotion Controllers for Quadrupedal Robots

2026-06-23 · David Rytz, Kim Tien Ly, Ioannis Havoutis arxiv

Universal quadrupedal locomotion remains limited by the difficulty of integrating perception across diverse robot morphologies. State-of-the-art controllers rely on single-robot training or blind policies that omit real-…

Reinforcement Learning

Differential Deep Detection in Massive MIMO With One-Bit ADC

2021-10-27 · Don-Roberts Emenonye, Carl Dietrich, R. Michael Buehrer

This article presents a differential detection scheme for the uplink of a massive MIMO system that employs one-bit quantizers on each receive antenna. We focus on the detection of differential amplitude and phase shift k…

Quantization

Preserving Expert-Level Privacy in Offline Reinforcement Learning

2024-11-18 · Navodita Sharma, Vishnu Vinod, Abhradeep Thakurta, Alekh Agarwal 외

The offline reinforcement learning (RL) problem aims to learn an optimal policy from historical data collected by one or more behavioural policies (experts) by interacting with an environment. However, the individual exp…

Offline RL reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Overcoming State Inertia in Full-Duplex Spoken Language Models via Activation Steering

2026-06-09 · Cheng-Kuang Chang, Kai-Wei Chang, Alexander H. Liu, James Glass arxiv

Full-duplex spoken language models (FD-SLMs) enable seamless speech interaction by allowing models to listen and speak simultaneously, yet the internal mechanism by which they coordinate listening and speaking remains un…