Temporal-Difference estimation of dynamic discrete choice models
We study the use of Temporal-Difference learning for estimating the structural parameters in dynamic discrete choice models. Our algorithms are based on the conditional choice probability approach but use functional approximations to estimate various terms in the pseudo-likelihood function. We suggest two approaches: The first - linear semi-gradient - provides approximations to the recursive terms using basis functions. The second - Approximate Value Iteration - builds a sequence of approximations to the recursive terms by solving non-parametric estimation problems. Our approaches are fast and naturally allow for continuous and/or high-dimensional state spaces. Furthermore, they do not require specification of transition densities. In dynamic games, they avoid integrating over other players' actions, further heightening the computational advantage. Our proposals can be paired with popular existing methods such as pseudo-maximum-likelihood, and we propose locally robust corrections for the latter to achieve parametric rates of convergence. Monte Carlo simulations confirm the properties of our algorithms in practice.
Code (0)
등록된 구현이 없습니다.
Tasks
Discrete Choice ModelsSimilar Papers 제목 키워드 기반
A Lecture Note on Offline RL and IRL, Part II: Foundations of Inverse Reinforcement Learning and Dynamic Discrete Choice Models
In the forward reinforcement-learning problem, the reward is fixed and known; the learner is asked to find a good policy or value function. Here we turn the question around. Given offline data generated by an expert, can…
Reinforcement LearningOffline RLDynamic Discrete-Continuous Choice Models: Identification and Conditional Choice Probability Estimation
This paper develops a general framework for dynamic models in which individuals simultaneously make both discrete and continuous choices. The framework incorporates a wide range of unobserved heterogeneity. I show that s…
quantile regressionA Non-Parametric Approach to Dynamic Programming
In this paper, we consider the problem of policy evaluation for continuous-state systems. We present a non-parametric approach to policy evaluation, which uses kernel density estimation to represent the system. The true …
Density EstimationNetwork-based Representations and Dynamic Discrete Choice Models for Multiple Discrete Choice Analysis
In many choice modeling applications, people demand is frequently characterized as multiple discrete, which means that people choose multiple items simultaneously. The analysis and prediction of people behavior in multip…
Discrete Choice ModelsMultiple-choiceA Recursive Partitioning Approach for Dynamic Discrete Choice Modeling in High Dimensional Settings
Dynamic discrete choice models are widely employed to answer substantive and policy questions in settings where individuals' current choices have future implications. However, estimation of these models is often computat…
Discrete Choice Models