paper-with-me

홈 › Papers

On the Benefits of Leveraging Structural Information in Planning Over the Learned Model

2023-03-15 · Jiajun Shen, Kananart Kuwaranancharoen, Raid Ayoub, Pietro Mercati, Shreyas Sundaram

Model-based Reinforcement Learning (RL) integrates learning and planning and has received increasing attention in recent years. However, learning the model can incur a significant cost (in terms of sample complexity), due to the need to obtain a sufficient number of samples for each state-action pair. In this paper, we investigate the benefits of leveraging structural information about the system in terms of reducing sample complexity. Specifically, we consider the setting where the transition probability matrix is a known function of a number of structural parameters, whose values are initially unknown. We then consider the problem of estimating those parameters based on the interactions with the environment. We characterize the difference between the Q estimates and the optimal Q value as a function of the number of samples. Our analysis shows that there can be a significant saving in sample complexity by leveraging structural information about the model. We illustrate the findings by considering several problems including controlling a queuing system with heterogeneous servers, and seeking an optimal path in a stochastic windy gridworld.

📄 PDF Abstract BibTeX arXiv:2303.08856

Code (0)

등록된 구현이 없습니다.

Tasks

Model-based Reinforcement LearningReinforcement Learning (RL)

Similar Papers 제목 키워드 기반

Mixture of Structural-and-Textual Retrieval over Text-rich Graph Knowledge Bases

2025-02-27 · Yongjia Lei, Haoyu Han, Ryan A. Rossi, Franck Dernoncourt 외

Text-rich Graph Knowledge Bases (TG-KBs) have become increasingly crucial for answering queries by providing textual and structural knowledge. However, current retrieval methods often retrieve these two types of knowledg…

RerankingRetrieval

SurgGoal: Rethinking Surgical Planning Evaluation via Goal-Satisfiability

2026-01-15 · Ruochen Li, Kun Yuan, Yufei Xia, Yue Zhou 외 arxiv

Surgical planning integrates visual perception, long-horizon reasoning, and procedural knowledge, yet it remains unclear whether current evaluation protocols reliably assess vision-language models (VLMs) in safety-critic…

A Framework for Neurosymbolic Robot Action Planning using Large Language Models

2023-03-01 · Alessio Capitanelli, Fulvio Mastrogiovanni

Symbolic task planning is a widely used approach to enforce robot autonomy due to its ease of understanding and deployment in robot architectures. However, techniques for symbolic task planning are difficult to scale in …

Task Planning

Structural hierarchical learning for energy networks

2023-02-08 · Julien Leprince, Waqas Khan, Henrik Madsen, Jan Kloppenborg Møller 외

Many sectors nowadays require accurate and coherent predictions across their organization to effectively operate. Otherwise, decision-makers would be planning using disparate views of the future, resulting in inconsisten…

Discrete Structural Planning for Generating Diverse Translations

2018-09-27 · Raphael Shu, Hideki Nakayama

Planning is important for humans when producing complex languages, which is a missing part in current language generation models. In this work, we add a planning phase in neural machine translation to control the global …

DiversityMachine TranslationSentenceText Generation+1