Automatic Induction of Bellman-Error Features for Probabilistic Planning

2014-01-16 · Jia-Hong Wu, Robert Givan

Domain-specific features are important in representing problem structure throughout machine learning and decision-theoretic planning. In planning, once state features are provided, domain-independent algorithms such as approximate value iteration can learn weighted combinations of those features that often perform well as heuristic estimates of state value (e.g., distance to the goal). Successful applications in real-world domains often require features crafted by human experts. Here, we propose automatic processes for learning useful domain-specific feature sets with little or no human intervention. Our methods select and add features that describe state-space regions of high inconsistency in the Bellman equation (statewise Bellman error) during approximate value iteration. Our method can be applied using any real-valued-feature hypothesis space and corresponding learning method for selecting features from training sets of state-value pairs. We evaluate the method with hypothesis spaces defined by both relational and propositional feature languages, using nine probabilistic planning domains. We show that approximate value iteration using a relational feature space performs at the state-of-the-art in domain-independent stochastic relational planning. Our method provides the first domain-independent approach that plays Tetris successfully (without human-engineered features).

📄 PDF Abstract BibTeX arXiv:1401.3847

Code (0)

등록된 구현이 없습니다.

Bellman Error Based Feature Generation using Random Projections on Sparse Spaces

2013-12-01 · NeurIPS 2013 12 · Mahdi Milani Fard, Yuri Grinberg, Amir-Massoud Farahmand, Joelle Pineau 외

This paper addresses the problem of automatic generation of features for value function approximation in reinforcement learning. Bellman Error Basis Functions (BEBFs) have been shown to improve the error of policy evalu…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Learning Concept Taxonomies from Multi-modal Data

2016-06-29 · ACL 2016 8 · Hao Zhang, Zhiting Hu, Yuntian Deng, Mrinmaya Sachan 외

We study the problem of automatically building hypernym taxonomies from textual and visual data. Previous works in taxonomy induction generally ignore the increasingly prominent visual data, which encode important percep…

Feature Engineering

Inference Over Programs That Make Predictions

2018-10-02 · Yura Perov

This abstract extends on the previous work (arXiv:1407.2646, arXiv:1606.00075) on program induction using probabilistic programming. It describes possible further steps to extend that work, such that, ultimately, automat…

Probabilistic Programming Program induction Program Synthesis

Automatic Taxonomy Induction and Expansion

2019-11-01 · IJCNLP 2019 11 · Nicolas Rodolfo Fauceglia, Alfio Gliozzo, Sarthak Dash, Md. Faisal Mahbub Chowdhury 외

The Knowledge Graph Induction Service (KGIS) is an end-to-end knowledge induction system. One of its main capabilities is to automatically induce taxonomies from input documents using a hybrid approach that takes advanta…

CertRL: Formalizing Convergence Proofs for Value and Policy Iteration in Coq

2020-09-23 · Koundinya Vajjha, Avraham Shinnar, Vasily Pestun, Barry Trager 외

Reinforcement learning algorithms solve sequential decision-making problems in probabilistic environments by optimizing for long-term reward. The desire to use reinforcement learning in safety-critical settings inspires …

Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1