Entrywise Inference for Missing Panel Data: A Simple and Instance-Optimal Approach

2024-01-24 · Yuling Yan, Martin J. Wainwright

Longitudinal or panel data can be represented as a matrix with rows indexed by units and columns indexed by time. We consider inferential questions associated with the missing data version of panel data induced by staggered adoption. We propose a computationally efficient procedure for estimation, involving only simple matrix algebra and singular value decomposition, and prove non-asymptotic and high-probability bounds on its error in estimating each missing entry. By controlling proximity to a suitably scaled Gaussian variable, we develop and analyze a data-driven procedure for constructing entrywise confidence intervals with pre-specified coverage. Despite its simplicity, our procedure turns out to be instance-optimal: we prove that the width of our confidence intervals match a non-asymptotic instance-wise lower bound derived via a Bayesian Cram\'{e}r-Rao argument. We illustrate the sharpness of our theoretical characterization on a variety of numerical examples. Our analysis is based on a general inferential toolbox for SVD-based algorithm applied to the matrix denoising model, which might be of independent interest.

📄 PDF Abstract BibTeX arXiv:2401.13665

Code (0)

등록된 구현이 없습니다.

Tasks

Causal Inference Denoising

Methods 이 논문이 사용한 방법론

Causal inference Causal inference is the process of drawing a conclusion about a causal connection based on the conditions of the occurrence of an effect. The main difference between causal…

Large Dimensional Latent Factor Modeling with Missing Observations and Applications to Causal Inference

2019-10-18 · Ruoxuan Xiong, Markus Pelger

This paper develops the inferential theory for latent factor models estimated from large dimensional panel data with missing observations. We propose an easy-to-use all-purpose estimator for a latent factor model by appl…

Causal Inference counterfactual Missing Values

A Robust Functional EM Algorithm for Incomplete Panel Count Data

2020-03-02 · NeurIPS 2020 12 · Alexander Moreno, Zhenke Wu, Jamie Yap, David Wetter 외

Panel count data describes aggregated counts of recurrent events observed at discrete time points. To understand dynamics of health behaviors, the field of quantitative behavioral research has evolved to increasingly rel…

Inference for Heteroskedastic PCA with Missing Data

2021-07-26 · Yuling Yan, Yuxin Chen, Jianqing Fan

This paper studies how to construct confidence regions for principal component analysis (PCA) in high dimension, a problem that has been vastly under-explored. While computing measures of uncertainty for nonlinear/noncon…

valid

Bias-Corrected Joint Spectral Embedding for Multilayer Networks with Invariant Subspace: Entrywise Eigenvector Perturbation and Inference

2024-06-12 · Fangzheng Xie

In this paper, we propose to estimate the invariant subspace across heterogeneous multiple networks using a novel bias-corrected joint spectral embedding algorithm. The proposed algorithm recursively calibrates the diago…

Community Detection

A Generalized Latent Factor Model Approach to Mixed-data Matrix Completion with Entrywise Consistency

2022-11-17 · Yunxiao Chen, Xiaoou Li

Matrix completion is a class of machine learning methods that concerns the prediction of missing entries in a partially observed matrix. This paper studies matrix completion for mixed data, i.e., data involving mixed typ…

Collaborative Filtering Matrix Completion