Time After Time: Deep-Q Effect Estimation for Interventions on When and What to do
Problems in fields such as healthcare, robotics, and finance requires reasoning about the value both of what decision or action to take and when to take it. The prevailing hope is that artificial intelligence will support such decisions by estimating the causal effect of policies such as how to treat patients or how to allocate resources over time. However, existing methods for estimating the effect of a policy struggle with \emph{irregular time}. They either discretize time, or disregard the effect of timing policies. We present a new deep-Q algorithm that estimates the effect of both when and what to do called Earliest Disagreement Q-Evaluation (EDQ). EDQ makes use of recursion for the Q-function that is compatible with flexible sequence models, such as transformers. EDQ provides accurate estimates under standard assumptions. We validate the approach through experiments on survival time and tumor growth tasks.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
A Practical Approach to Causal Inference over Time
In this paper, we focus on estimating the causal effect of an intervention over time on a dynamical system. To that end, we formally define causal interventions and their effects over time on discrete-time stochastic pro…
Causal InferenceEconometricsWhen Should We Introduce Safety Interventions During Pretraining?
Prior work has shown that safety interventions applied during pretraining, such as removing and rephrasing harmful content, can substantially improve the robustness of the resulting models. In this paper, we study the fu…
Rapidly evaluating lockdown strategies using spectral analysis: the cycles behind new daily COVID-19 cases and what happens after lockdown
Spectral analysis characterises oscillatory time series behaviours such as cycles, but accurate estimation requires reasonable numbers of observations. Current COVID-19 time series for many countries are short: pre- and …
Time SeriesTime Series AnalysisCOVID-19 in Africa -- outbreak despite interventions?
Few African countries have reported COVID-19 case numbers above $1\,000$ as of April 18, 2020, with South Africa reporting $3\,034$ cases being hit hardest in Sub-Saharan Africa. Several African countries, especially Sou…
SigmaScheduling: Uncertainty-Informed Scheduling of Decision Points for Intelligent Mobile Health Interventions
Timely decision making is critical to the effectiveness of mobile health (mHealth) interventions. At predefined timepoints called "decision points," intelligent mHealth systems such as just-in-time adaptive interventions…
Decision Making