Towards Mixed Optimization for Reinforcement Learning with Program Synthesis

2018-07-01 · Surya Bhupatiraju, Kumar Krishna Agrawal, Rishabh Singh

Deep reinforcement learning has led to several recent breakthroughs, though the learned policies are often based on black-box neural networks. This makes them difficult to interpret and to impose desired specification constraints during learning. We present an iterative framework, MORL, for improving the learned policies using program synthesis. Concretely, we propose to use synthesis techniques to obtain a symbolic representation of the learned policy, which can then be debugged manually or automatically using program repair. After the repair step, we use behavior cloning to obtain the policy corresponding to the repaired program, which is then further improved using gradient descent. This process continues until the learned policy satisfies desired constraints. We instantiate MORL for the simple CartPole problem and show that the programmatic representation allows for high-level modifications that in turn lead to improved learning of the policies.

📄 PDF Abstract BibTeX arXiv:1807.00403

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement LearningProgram Repair Program Synthesis reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Hexcute: A Tile-based Programming Language with Automatic Layout and Task-Mapping Synthesis

2025-04-22 · Xiao Zhang, Yaoyao Ding, Yang Hu, Gennady Pekhimenko

Deep learning (DL) workloads mainly run on accelerators like GPUs. Recent DL quantization techniques demand a new matrix multiplication operator with mixed input data types, further complicating GPU optimization. Prior h…

GPU Quantization

Integrating Reinforcement Learning and Model Predictive Control with Applications to Microgrids

2024-09-17 · Caio Fabio Oliveira da Silva, Azita Dabiri, Bart De Schutter

This work proposes an approach that integrates reinforcement learning and model predictive control (MPC) to solve finite-horizon optimal control problems in mixed-logical dynamical systems efficiently. Optimization-based…

Model Predictive Control reinforcement-learning Reinforcement Learning

Accelerating Cutting-Plane Algorithms via Reinforcement Learning Surrogates

2023-07-17 · Kyle Mana, Fernando Acero, Stephen Mak, Parisa Zehtabi 외

Discrete optimization belongs to the set of $\mathcal{NP}$-hard problems, spanning fields such as mixed-integer programming and combinatorial optimization. A current standard approach to solving convex discrete optimizat…

Combinatorial Optimization Management reinforcement-learning Reinforcement Learning+1

GPU-Accelerated Synthesis of Mixed-Boolean Arithmetic: Beyond Caching

2026-05-07 · Gabriel Bathie, Baptiste Mouillon, Nathanaël Fijalkow arxiv

Synthesizing Mixed-Boolean Arithmetic (MBA) expressions from input-output examples is central to program deobfuscation and also useful for compiler optimization, reverse engineering, and cryptanalysis. Existing MBA synth…

A Reinforcement Learning Environment for Mathematical Reasoning via Program Synthesis

2021-07-15 · Joseph Palermo, Johnny Ye, Alok Singh

We convert the DeepMind Mathematics Dataset into a reinforcement learning environment by interpreting it as a program synthesis problem. Each action taken in the environment adds an operator or an input into a discrete c…

Mathematical Reasoning Program Synthesis reinforcement-learning Reinforcement Learning+1