Large Scale Longitudinal Experiments: Estimation and Inference
Large-scale randomized experiments are seldom analyzed using panel regression methods because of computational challenges arising from the presence of millions of nuisance parameters. We leverage Mundlak's insight that unit intercepts can be eliminated by using carefully chosen averages of the regressors to rewrite several common estimators in a form that is amenable to weighted-least squares estimation with frequency weights. This renders regressions involving arbitrary strata intercepts tractable with very large datasets, optionally with the key compression step computed out-of-memory in SQL. We demonstrate that these methods yield more precise estimates than other commonly used estimators, and also find that the compression strategy greatly increases computational efficiency. We provide in-memory (pyfixest) and out-of-memory (duckreg) python libraries to implement these estimators.
Code (1)
Tasks
Computational EfficiencySimilar Papers 제목 키워드 기반
Scalable Counterfactual Risk Estimation for Rare Events in Longitudinal Data
Estimating the causal effect of time-varying treatments on survival outcomes in large observational studies is computationally demanding, particularly when outcomes are rare. While g-formula-based methods such as the ite…
Causal InferenceInstrumental Variable Estimation for Causal Inference in Longitudinal Data with Time-Dependent Latent Confounders
Causal inference from longitudinal observational data is a challenging problem due to the difficulty in correctly identifying the time-dependent confounders, especially in the presence of latent time-dependent confounder…
Causal InferenceDifferentially Private Inference for Longitudinal Linear Regression
Differential Privacy (DP) provides a rigorous framework for releasing statistics while protecting individual information present in a dataset. Although substantial progress has been made on differentially private linear …
Statistical Inference After Adaptive Sampling for Longitudinal Data
Online reinforcement learning and other adaptive sampling algorithms are increasingly used in digital intervention experiments to optimize treatment delivery for users over time. In this work, we focus on longitudinal us…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Modeling Covariate Transition for Efficient Estimation of Longitudinal Treatment Effects in Randomized Experiments
We present a regression-adjustment framework designed for the estimation of longitudinal treatment effects in randomized experiments under static regimes. While regression-adjustment methods are useful for variance reduc…