Improving the Performance of Backward Chained Behavior Trees that use Reinforcement Learning
In this letter we show how to improve the performance of backward chained behavior trees (BTs) that use reinforcement learning (RL). BTs represent a hierarchical and modular way of combining control policies into higher level control policies. Backward chaining is a design principle for the construction of BTs that combine reactivity with goal directed actions in a structured way. The backward chained structure has also enabled convergence proofs for BTs, identifying a set of local conditions that lead to the convergence of all trajectories to a set of desired goal states. The key idea of this letter is to improve performance of backward chained BTs by using the conditions identified in a theoretical convergence proof to setup the RL problems for individual controllers. In particular, previous analysis identified so-called active constraint conditions (ACCs), that should not be broken in order to avoid having to return to work on previously achieved subgoals. We propose a way to setup the RL problems, such that they do not only achieve each immediate subgoal, but also avoid violating the identified ACCs. The resulting performance improvement depends on how often ACC violations occurred before the change, and how much effort was needed to re-achieve them. The proposed approach is illustrated in a dynamic simulation environment.
Code (1)
Tasks
reinforcement-learningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Scheherazade: Evaluating Chain-of-Thought Math Reasoning in LLMs with Chain-of-Problems
Benchmarks are critical for measuring progress of math reasoning abilities of Large Language Models (LLMs). However, existing widely-used benchmarks such as GSM8K have been rendered less useful as multiple cutting-edge L…
GSM8KMathMathematical ReasoningBackward Imitation and Forward Reinforcement Learning via Bi-directional Model Rollouts
Traditional model-based reinforcement learning (RL) methods generate forward rollout traces using the learnt dynamics model to reduce interactions with the real environment. The recent model-based RL method considers the…
Generative Adversarial NetworkModel-based Reinforcement LearningMuJoCoreinforcement-learning+1Adding Neural Network Controllers to Behavior Trees without Destroying Performance Guarantees
In this paper, we show how Behavior Trees that have performance guarantees, in terms of safety and goal convergence, can be extended with components that were designed using machine learning, without destroying those per…
BIG-bench Machine LearningReinforcement LearningA Survey of In-Context Reinforcement Learning
Reinforcement learning (RL) agents typically optimize their policies by performing expensive backward passes to update their network parameters. However, some agents can solve new tasks without updating any parameters by…
In-Context Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1A Framework for Constrained and Adaptive Behavior-Based Agents
Behavior Trees are commonly used to model agents for robotics and games, where constrained behaviors must be designed by human experts in order to guarantee that these agents will execute a specific chain of actions give…
Hierarchical Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)