paper-with-me

홈 › Papers

Distilling Reinforcement Learning into Single-Batch Datasets

2025-08-12 · Connor Wilhelm, Dan Ventura arxiv

Dataset distillation compresses a large dataset into a small synthetic dataset such that learning on the synthetic dataset approximates learning on the original. Training on the distilled dataset can be performed in as little as one step of gradient descent. We demonstrate that distillation is generalizable to different tasks by distilling reinforcement learning environments into one-batch supervised learning datasets. This demonstrates not only distillation's ability to compress a reinforcement learning task but also its ability to transform one learning modality (reinforcement learning) into another (supervised learning). We present a novel extension of proximal policy optimization for meta-learning and use it in distillation of a multi-dimensional extension of the classic cart-pole problem, all MuJoCo environments, and several Atari games. We demonstrate distillation's ability to compress complex RL environments into one-step supervised learning, explore RL distillation's generalizability across learner architectures, and demonstrate distilling an environment into the smallest-possible synthetic dataset.

📄 PDF Abstract BibTeX arXiv:2508.09283

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement LearningAtari Games

Similar Papers 제목 키워드 기반

Distilling Ensembles Improves Uncertainty Estimates

2020-11-23 · pproximateinference AABI Symposium 2021 1 · Zelda E Mariet, Rodolphe Jenatton, Florian Wenzel, Dustin Tran

We seek to bridge the performance gap between batch ensembles (ensembles of deep networks with shared parameters) and deep ensembles on tasks which require not only predictions, but also uncertainty estimates for these …

Benchmarking Sample Selection Strategies for Batch Reinforcement Learning

2021-09-29 · Yuwei Fu, Di wu, Benoit Boulet

Training sample section techniques, such as prioritized experience replay (PER), have been recognized as of significant importance for online reinforcement learning algorithms. Efficient sample selection can help further…

BenchmarkingImitation Learningreinforcement-learningReinforcement Learning+1

Stability and Generalization for Bellman Residuals

2025-08-26 · Enoch H. Kang, Kyoungseok Jang arxiv

Offline reinforcement learning and offline inverse reinforcement learning aim to recover near-optimal value functions or reward models from a fixed batch of logged trajectories, yet current practice still struggles to en…

Reinforcement Learning

Large Batch Simulation for Deep Reinforcement Learning

2021-03-12 · ICLR 2021 1 · Brennan Shacklett, Erik Wijmans, Aleksei Petrenko, Manolis Savva 외

We accelerate deep reinforcement learning-based training in visually complex 3D environments by two orders of magnitude over prior work, realizing end-to-end training speeds of over 19,000 frames of experience per second…

Deep Reinforcement LearningGPUPointGoal Navigationreinforcement-learning+2

Adversarial Score identity Distillation: Rapidly Surpassing the Teacher in One Step

2024-10-19 · Mingyuan Zhou, Huangjie Zheng, Yi Gu, Zhendong Wang 외

Score identity Distillation (SiD) is a data-free method that has achieved SOTA performance in image generation by leveraging only a pretrained diffusion model, without requiring any training data. However, its ultimate p…

Conditional Image GenerationGPUImage GenerationUnconditional Image Generation