Model-based Meta Reinforcement Learning using Graph Structured Surrogate Models
Reinforcement learning is a promising paradigm for solving sequential decision-making problems, but low data efficiency and weak generalization across tasks are bottlenecks in real-world applications. Model-based meta reinforcement learning addresses these issues by learning dynamics and leveraging knowledge from prior experience. In this paper, we take a closer look at this framework, and propose a new Thompson-sampling based approach that consists of a new model to identify task dynamics together with an amortized policy optimization step. We show that our model, called a graph structured surrogate model (GSSM), outperforms state-of-the-art methods in predicting environment dynamics. Additionally, our approach is able to obtain high returns, while allowing fast execution during deployment by avoiding test time policy gradient optimization.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingMeta Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Sequential Decision MakingThompson SamplingSimilar Papers 제목 키워드 기반
Surrogate Learning in Meta-Black-Box Optimization: A Preliminary Study
Recent Meta-Black-Box Optimization (MetaBBO) approaches have shown possibility of enhancing the optimization performance through learning meta-level policies to dynamically configure low-level optimizers. However, existi…
Kolmogorov-Arnold NetworksReinforcement Learning (RL)Debiased Graph Poisoning Attack via Contrastive Surrogate Objective
Graph neural networks (GNN) are vulnerable to adversarial attacks, which aim to degrade the performance of GNNs through imperceptible changes on the graph. However, we find that in fact the prevalent meta-gradient-based …
Encoding Metal Mask Projection for Metal Artifact Reduction in Computed Tomography
Metal artifact reduction (MAR) in computed tomography (CT) is a notoriously challenging task because the artifacts are structured and non-local in the image domain. However, they are inherently local in the sinogram doma…
Computed Tomography (CT)Metal Artifact ReductionAdaptive Constraint Propagation: Scaling Structured Inference for Large Language Models via Meta-Reinforcement Learning
Large language models increasingly require structured inference, from JSON schema enforcement to multi-lingual parsing, where outputs must satisfy complex constraints. We introduce MetaJuLS, a meta-reinforcement learning…
Reinforcement LearningDomain AdaptationMeta-Black-Box Optimization with Ensemble Surrogate Modeling for Robustness-Accuracy Trade-off within SAEA
Surrogate-assisted evolutionary algorithms (SAEAs) have been widely used for expensive black-box optimization problems. However, their reliance on rigid and manually designed components limits their flexibility and gener…
Reinforcement LearningEnsemble Learning