Redesigning Regularization for Effective Policy Smoothing

2026-06-11 · Taisuke Kobayashi, Naoto Yamanaka arxiv

This paper proposes a novel regularization design to effectively smooth policy functions in reinforcement learning. While regularization that enhances `global'' Lipschitz continuity was initially considered, it has been limited to `local'' Lipschitz continuity due to a tradeoff between smoothness and expressiveness. However, it has become apparent that the original implementation is cumbersome and does not provide sufficient smoothing, leading to a preference for simpler implementations. This stems from a discrepancy between theory and implementation, and a more appropriate implementation can expect to facilitate smoothing. Therefore, this paper identifies three reasons why the original implementation does not function adequately and provide remedies for them. This modified regularization performs well across multiple tasks and algorithms, successfully achieving smooth motion while improving control performance. Furthermore, by applying it to sim-to-real reinforcement learning for a quadruped robot, it is demonstrated that smooth motion provides robustness against sudden changes in target velocity commands.

📄 PDF Abstract BibTeX arXiv:2606.13169

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

Laplacian-LoRA: Delaying Oversmoothing in Deep GCNs via Spectral Low-Rank Adaptation

2026-02-07 · Sai Vamsi Alisetti arxiv

Oversmoothing is a fundamental limitation of deep graph convolutional networks (GCNs), causing node representations to collapse as depth increases. While many prior approaches mitigate this effect through architectural m…

Efficient Preference-based Reinforcement Learning via Aligned Experience Estimation

2024-05-29 · Fengshuo Bai, Rui Zhao, Hongming Zhang, Sijia Cui 외

Preference-based reinforcement learning (PbRL) has shown impressive capabilities in training agents without reward engineering. However, a notable limitation of PbRL is its dependency on substantial human feedback. This …

reinforcement-learning Reinforcement Learning

Regularization via Structural Label Smoothing

2020-01-07 · Weizhi Li, Gautam Dasarathy, Visar Berisha

Regularization is an effective way to promote the generalization performance of machine learning models. In this paper, we focus on label smoothing, a form of output distribution regularization that prevents overfitting …

Random Smoothing Regularization in Kernel Gradient Descent Learning

2023-05-05 · Liang Ding, Tianyang Hu, Jiahang Jiang, Donghao Li 외

Random smoothing data augmentation is a unique form of regularization that can prevent overfitting by introducing noise to the input data, encouraging the model to learn more generalized features. Despite its success in …

Data Augmentation

Regularization of the policy updates for stabilizing Mean Field Games

2023-04-04 · Talal Algumaei, Ruben Solozabal, REDA ALAMI, Hakim Hacid 외

This work studies non-cooperative Multi-Agent Reinforcement Learning (MARL) where multiple agents interact in the same environment and whose goal is to maximize the individual returns. Challenges arise when scaling up th…

Deep Reinforcement Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning