Model-based Policy Optimization using Symbolic World Model

2024-07-18 · Andrey Gorodetskiy, Konstantin Mironov, Aleksandr Panov

The application of learning-based control methods in robotics presents significant challenges. One is that model-free reinforcement learning algorithms use observation data with low sample efficiency. To address this challenge, a prevalent approach is model-based reinforcement learning, which involves employing an environment dynamics model. We suggest approximating transition dynamics with symbolic expressions, which are generated via symbolic regression. Approximation of a mechanical system with a symbolic model has fewer parameters than approximation with neural networks, which can potentially lead to higher accuracy and quality of extrapolation. We use a symbolic dynamics model to generate trajectories in model-based policy optimization to improve the sample efficiency of the learning algorithm. We evaluate our approach across various tasks within simulated environments. Our method demonstrates superior sample efficiency in these tasks compared to model-free and model-based baseline methods.

📄 PDF Abstract BibTeX arXiv:2407.13518

Code (0)

등록된 구현이 없습니다.

Tasks

modelModel-based Reinforcement Learningreinforcement-learning Reinforcement LearningSymbolic Regression

Sample-efficient Neuro-symbolic Proximal Policy Optimization

2026-04-28 · Simone Murari, Celeste Veronese, Daniele Meli arxiv

Deep Reinforcement Learning (DRL) algorithms often require a large amount of data and struggle in sparse-reward domains with long planning horizons and multiple sub-goals. In this paper, we propose a neuro-symbolic exten…

Reinforcement Learning

Neuro-Symbolic World Models for Adapting to Open World Novelty

2023-01-16 · Jonathan Balloch, Zhiyu Lin, Robert Wright, Xiangyu Peng 외

Open-world novelty--a sudden change in the mechanics or properties of an environment--is a common occurrence in the real world. Novelty adaptation is an agent's ability to improve its policy performance post-novelty. Mos…

Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Neurosymbolic Reinforcement Learning with Formally Verified Exploration

2020-09-26 · NeurIPS 2020 12 · Greg Anderson, Abhinav Verma, Isil Dillig, Swarat Chaudhuri

We present Revel, a partially neural reinforcement learning (RL) framework for provably safe exploration in continuous state and action spaces. A key challenge for provably safe deep RL is that repeatedly verifying neura…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Safe Exploration

DiSProD: Differentiable Symbolic Propagation of Distributions for Planning

2023-02-03 · Palash Chatterjee, Ashutosh Chapagain, Weizhe Chen, Roni Khardon

The paper introduces DiSProD, an online planner developed for environments with probabilistic transitions in continuous state and action spaces. DiSProD builds a symbolic graph that captures the distribution of future tr…

Navigate

Deep Symbolic Optimization: Reinforcement Learning for Symbolic Mathematics

2025-05-16 · Conor F. Hayes, Felipe Leno Da Silva, Jiachen Yang, T. Nathan Mundhenk 외

Deep Symbolic Optimization (DSO) is a novel computational framework that enables symbolic optimization for scientific discovery, particularly in applications involving the search for intricate symbolic structures. One no…

Equation Discovery reinforcement-learning Reinforcement Learning scientific discovery+1