Meta-Q-Learning
This paper introduces Meta-Q-Learning (MQL), a new off-policy algorithm for meta-Reinforcement Learning (meta-RL). MQL builds upon three simple ideas. First, we show that Q-learning is competitive with state-of-the-art meta-RL algorithms if given access to a context variable that is a representation of the past trajectory. Second, a multi-task objective to maximize the average reward across the training tasks is an effective method to meta-train RL policies. Third, past data from the meta-training replay buffer can be recycled to adapt the policy on a new task using off-policy updates. MQL draws upon ideas in propensity estimation to do so and thereby amplifies the amount of available data for adaptation. Experiments on standard continuous-control benchmarks suggest that MQL compares favorably with the state of the art in meta-RL.
Code (2)
Tasks
continuous-controlContinuous ControlMeta Reinforcement LearningQ-LearningReinforcement LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
DAC-MR: Data Augmentation Consistency Based Meta-Regularization for Meta-Learning
Meta learning recently has been heavily researched and helped advance the contemporary machine learning. However, achieving well-performing meta-learning model requires a large amount of training tasks with high-quality …
Data AugmentationMeta-LearningMeta-Interpretive Learning as Metarule Specialisation
In Meta-Interpretive Learning (MIL) the metarules, second-order datalog clauses acting as inductive bias, are manually defined by the user. In this work we show that second-order metarules for MIL can be learned by MIL. …
Inductive BiasNominal Metaphor Generation with Multitask Learning
Metaphor generation is a challenging task which can impact many downstream tasks such as improving user satisfaction with dialogue systems and story generation. This paper tackles the problem of Chinese nominal metaphor …
Story GenerationMetaFill: Text Infilling for Meta-Path Generation on Heterogeneous Information Networks
Heterogeneous Information Network (HIN) is essential to study complicated networks containing multiple edge types and node types. Meta-path, a sequence of node types and edge types, is the core technique to embed HINs. S…
Graph EmbeddingLanguage ModellingLink PredictionNode Classification+1Meta-node: A Concise Approach to Effectively Learn Complex Relationships in Heterogeneous Graphs
Existing message passing neural networks for heterogeneous graphs rely on the concepts of meta-paths or meta-graphs due to the intrinsic nature of heterogeneous graphs. However, the meta-paths and meta-graphs need to be …
Contrastive LearningGraph LearningNode Clustering