paper-with-me

홈 › Papers

Doubly Robust Off-Policy Learning on Low-Dimensional Manifolds by Deep Neural Networks

2020-11-03 · Minshuo Chen, Hao liu, Wenjing Liao, Tuo Zhao

Causal inference explores the causation between actions and the consequent rewards on a covariate set. Recently deep learning has achieved a remarkable performance in causal inference, but existing statistical theories cannot well explain such an empirical success, especially when the covariates are high-dimensional. Most theoretical results in causal inference are asymptotic, suffer from the curse of dimensionality, and only work for the finite-action scenario. To bridge such a gap between theory and practice, this paper studies doubly robust off-policy learning by deep neural networks. When the covariates lie on a low-dimensional manifold, we prove nonasymptotic regret bounds, which converge at a fast rate depending on the intrinsic dimension of the manifold. Our results cover both the finite- and continuous-action scenarios. Our theory shows that deep neural networks are adaptive to the low-dimensional geometric structures of the covariates, and partially explains the success of deep learning for causal inference.

📄 PDF Abstract BibTeX arXiv:2011.01797

Code (0)

등록된 구현이 없습니다.

Tasks

Causal Inference

Methods 이 논문이 사용한 방법론

Causal inference Causal inference is the process of drawing a conclusion about a causal connection based on the conditions of the occurrence of an effect. The main difference between causal…

Similar Papers 제목 키워드 기반

Doubly Robust Off-Policy Actor-Critic Algorithms for Reinforcement Learning

2019-12-11 · Riashat Islam, Raihan Seraj, Samin Yeasar Arnob, Doina Precup

We study the problem of off-policy critic evaluation in several variants of value-based off-policy actor-critic algorithms. Off-policy actor-critic algorithms require an off-policy critic evaluation step, to estimate the…

continuous-controlContinuous Controlreinforcement-learningReinforcement Learning+2

System stabilization with policy optimization on unstable latent manifolds

2024-07-08 · Steffen W. R. Werner, Benjamin Peherstorfer

Stability is a basic requirement when studying the behavior of dynamical systems. However, stabilizing dynamical systems via reinforcement learning is challenging because only little data can be collected over short time…

reinforcement-learningReinforcement Learning

Unsupervised Manifold Linearizing and Clustering

2023-01-04 · ICCV 2023 1 · Tianjiao Ding, Shengbang Tong, Kwan Ho Ryan Chan, Xili Dai 외

We consider the problem of simultaneously clustering and learning a linear representation of data lying close to a union of low-dimensional manifolds, a fundamental task in machine learning and computer vision. When the …

ClusteringDeep Clustering

Doubly Robust Policy Evaluation and Learning

2011-03-23 · Miroslav Dudik, John Langford, Lihong Li

We study decision making in environments where the reward is only partially observed, but can be modeled as a function of an action and an observed context. This setting, known as contextual bandits, encompasses a wide v…

Decision MakingMulti-Armed Bandits

Doubly Robust Off-Policy Value and Gradient Estimation for Deterministic Policies

2020-06-06 · NeurIPS 2020 12 · Nathan Kallus, Masatoshi Uehara

Offline reinforcement learning, wherein one uses off-policy data logged by a fixed behavior policy to evaluate and learn new policies, is crucial in applications where experimentation is limited such as medicine. We stud…