Program-Based Strategy Induction for Reinforcement Learning
Typical models of learning assume incremental estimation of continuously-varying decision variables like expected rewards. However, this class of models fails to capture more idiosyncratic, discrete heuristics and strategies that people and animals appear to exhibit. Despite recent advances in strategy discovery using tools like recurrent networks that generalize the classic models, the resulting strategies are often onerous to interpret, making connections to cognition difficult to establish. We use Bayesian program induction to discover strategies implemented by programs, letting the simplicity of strategies trade off against their effectiveness. Focusing on bandit tasks, we find strategies that are difficult or unexpected with classical incremental learning, like asymmetric learning from rewarded and unrewarded trials, adaptive horizon-dependent random exploration, and discrete state switching.
Code (0)
등록된 구현이 없습니다.
Tasks
Incremental LearningProgram inductionreinforcement-learningReinforcement LearningSimilar Papers 제목 키워드 기반
Inferential Induction: A Novel Framework for Bayesian Reinforcement Learning
Bayesian reinforcement learning (BRL) offers a decision-theoretic solution for reinforcement learning. While "model-based" BRL algorithms have focused either on maintaining a posterior distribution on models or value fun…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Few-Shot Complex Knowledge Base Question Answering via Meta Reinforcement Learning
Complex question-answering (CQA) involves answering complex natural-language questions on a knowledge base (KB). However, the conventional neural program induction (NPI) approach exhibits uneven performance when the ques…
Knowledge Base Question AnsweringMeta Reinforcement LearningProgram inductionQuestion Answering+2Neural Program Meta-Induction
Most recently proposed methods for Neural Program Induction work under the assumption of having a large set of input/output (I/O) examples for learning any underlying input-output mapping. This paper aims to address the …
Program inductionTransfer LearningUsing Natural Language and Program Abstractions to Instill Human Inductive Biases in Machines
Strong inductive biases give humans the ability to quickly learn to perform a variety of tasks. Although meta-learning is a method to endow neural networks with useful inductive biases, agents trained by meta-learning ma…
Meta-LearningMeta Reinforcement LearningProgram inductionReinforcement Learning (RL)Guided Sketch-Based Program Induction by Search Gradients
Many tasks can be easily solved using machine learning techniques. However, some tasks cannot readily be solved using statistical models, requiring a symbolic approach instead. Program induction is one of the ways that s…
Program induction