Is Conditional Generative Modeling all you need for Decision-Making?

2022-11-28 · Anurag Ajay, Yilun Du, Abhi Gupta, Joshua Tenenbaum, Tommi Jaakkola, Pulkit Agrawal

Recent improvements in conditional generative modeling have made it possible to generate high-quality images from language descriptions alone. We investigate whether these methods can directly address the problem of sequential decision-making. We view decision-making not through the lens of reinforcement learning (RL), but rather through conditional generative modeling. To our surprise, we find that our formulation leads to policies that can outperform existing offline RL approaches across standard benchmarks. By modeling a policy as a return-conditional diffusion model, we illustrate how we may circumvent the need for dynamic programming and subsequently eliminate many of the complexities that come with traditional offline RL. We further demonstrate the advantages of modeling policies as conditional diffusion models by considering two other conditioning variables: constraints and skills. Conditioning on a single constraint or skill during training leads to behaviors at test-time that can satisfy several constraints together or demonstrate a composition of skills. Our results illustrate that conditional generative modeling is a powerful tool for decision-making.

📄 PDF Abstract BibTeX arXiv:2211.15657

Code (0)

등록된 구현이 없습니다.

Tasks

AllDecision Making Offline RL Reinforcement Learning (RL)Sequential Decision Making

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Learning to Simulate: Generative Metamodeling via Quantile Regression

2023-11-29 · L. Jeff Hong, Yanxi Hou, Qingkai Zhang, Xiaowei Zhang

Stochastic simulation models effectively capture complex system dynamics but are often too slow for real-time decision-making. Traditional metamodeling techniques learn relationships between simulator inputs and a single…

Decision Making quantile regression regression

GenAI4UQ: A Software for Inverse Uncertainty Quantification Using Conditional Generative Models

2024-12-09 · Ming Fan, Zezhong Zhang, Dan Lu, Guannan Zhang

We introduce GenAI4UQ, a software package for inverse uncertainty quantification in model calibration, parameter estimation, and ensemble forecasting in scientific applications. GenAI4UQ leverages a generative artificial…

parameter estimation Uncertainty Quantification

Federated Learning for Non-factorizable Models using Deep Generative Prior Approximations

2024-05-25 · Conor Hassan, Joshua J Bon, Elizaveta Semenova, Antonietta Mira 외

Federated learning (FL) allows for collaborative model training across decentralized clients while preserving privacy by avoiding data sharing. However, current FL methods assume conditional independence between client m…

Decision Making Epidemiology Federated Learning Gaussian Processes

Generative Latent Diffusion Model for Inverse Modeling and Uncertainty Analysis in Geological Carbon Sequestration

2025-08-17 · Zhao Feng, Xin-Yang Liu, Meet Hemant Parikh, Junyi Guo 외 arxiv

Geological Carbon Sequestration (GCS) has emerged as a promising strategy for mitigating global warming, yet its effectiveness heavily depends on accurately characterizing subsurface flow dynamics. The inherent geologica…

Graphical conditional generative modeling for digital twin modeling

2026-06-15 · Zongren Zou, Théo Bourdais, Ricardo Baptista, Houman Owhadi arxiv

Digital twin modeling, including control and data assimilation under model uncertainty, often faces an open-ended fidelity problem: adding variables, data streams, and time scales can indefinitely increase model complexi…

Reinforcement Learning