Distributed Nonparametric Estimation: from Sparse to Dense Samples per Terminal
Consider the communication-constrained problem of nonparametric function estimation, in which each distributed terminal holds multiple i.i.d. samples. Under certain regularity assumptions, we characterize the minimax optimal rates for all regimes, and identify phase transitions of the optimal rates as the samples per terminal vary from sparse to dense. This fully solves the problem left open by previous works, whose scopes are limited to regimes with either dense samples or a single sample per terminal. To achieve the optimal rates, we design a layered estimation protocol by exploiting protocols for the parametric density estimation problem. We show the optimality of the protocol using information-theoretic methods and strong data processing inequalities, and incorporating the classic balls and bins model. The optimal rates are immediate for various special cases such as density estimation, Gaussian, binary, Poisson and heteroskedastic regression models.
Code (0)
등록된 구현이 없습니다.
Tasks
Density EstimationSimilar Papers 제목 키워드 기반
Optimal Federated Learning for Nonparametric Regression with Heterogeneous Distributed Differential Privacy Constraints
This paper studies federated learning for nonparametric regression in the context of distributed samples across different servers, each adhering to distinct differential privacy constraints. The setting we consider is he…
Federated LearningPrivacy PreservingregressionDebiased distributed learning for sparse partial linear models in high dimensions
Although various distributed machine learning schemes have been proposed recently for pure linear models and fully nonparametric models, little attention has been paid on distributed optimization for semi-paramemetric mo…
Distributed OptimizationVocal Bursts Intensity PredictionInformation-Theoretic Bounds for Sparse Covariance Estimation in the Vertical-Split Distributed Model
We study the minimax estimation error for distributed covariance matrix estimation in the vertical-split (feature-split) setting, where two agents each observe different coordinates of $m$ i.i.d. sub-Gaussian samples and…
Distributed Nonparametric Function Estimation: Optimal Rate of Convergence and Cost of Adaptation
Distributed minimax estimation and distributed adaptive estimation under communication constraints for Gaussian sequence model and white noise model are studied. The minimax rate of convergence for distributed estimation…
Sparse deep neural networks for nonparametric estimation in high-dimensional sparse regression
Generalization theory has been established for sparse deep neural networks under high-dimensional regime. Beyond generalization, parameter estimation is also important since it is crucial for variable selection and inter…
parameter estimationregressionVariable Selection