In Defense of the Indefensible: A Very Naive Approach to High-Dimensional Inference
A great deal of interest has recently focused on conducting inference on the parameters in a high-dimensional linear model. In this paper, we consider a simple and very na\"{i}ve two-step procedure for this task, in which we (i) fit a lasso model in order to obtain a subset of the variables, and (ii) fit a least squares model on the lasso-selected set. Conventional statistical wisdom tells us that we cannot make use of the standard statistical inference tools for the resulting least squares model (such as confidence intervals and $p$-values), since we peeked at the data twice: once in running the lasso, and again in fitting the least squares model. However, in this paper, we show that under a certain set of assumptions, with high probability, the set of variables selected by the lasso is identical to the one selected by the noiseless lasso and is hence deterministic. Consequently, the na\"{i}ve two-step approach can yield asymptotically valid inference. We utilize this finding to develop the \emph{na\"ive confidence interval}, which can be used to draw inference on the regression coefficients of the model selected by the lasso, as well as the \emph{na\"ive score test}, which can be used to test the hypotheses regarding the full-model regression coefficients.
Code (0)
등록된 구현이 없습니다.
Tasks
regressionvalidSimilar Papers 제목 키워드 기반
On the Robustness of Deep K-Nearest Neighbors
Despite a large amount of attention on adversarial examples, very few works have demonstrated an effective defense against this threat. We examine Deep k-Nearest Neighbor (DkNN), a proposed defense that combines k-Neares…
Sequential Randomized Smoothing for Adversarially Robust Speech Recognition
While Automatic Speech Recognition has been shown to be vulnerable to adversarial attacks, defenses against these attacks are still lagging. Existing, naive defenses can be partially broken with an adaptive attack. In cl…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Robust Speech Recognitionspeech-recognition+1A Mixture Model Based Defense for Data Poisoning Attacks Against Naive Bayes Spam Filters
Naive Bayes spam filters are highly susceptible to data poisoning attacks. Here, known spam sources/blacklisted IPs exploit the fact that their received emails will be treated as (ground truth) labeled spam examples, and…
Data PoisoningPoisoning the Watchtower: Prompt Injection Attacks Against LLM-Augmented Security Operations Through Adversarial Log Content
Large language models (LLMs) are increasingly used as analyst assistants in security operations centers (SOCs), where they ingest log and alert data to produce triage labels, incident summaries, or remediation advice. We…
DistillGuard: Evaluating Defenses Against LLM Knowledge Distillation
Knowledge distillation from proprietary LLM APIs poses a growing threat to model providers, yet defenses against this attack remain fragmented and unevaluated. We present DistillGuard, a framework for systematically eval…
Knowledge DistillationMathematical ReasoningCode Generation