Leveraging Spatial and Temporal Correlations in Sparsified Mean Estimation

2021-10-14 · NeurIPS 2021 12 · Divyansh Jhunjhunwala, Ankur Mallick, Advait Gadhikar, Swanand Kadhe, Gauri Joshi

We study the problem of estimating at a central server the mean of a set of vectors distributed across several nodes (one vector per node). When the vectors are high-dimensional, the communication cost of sending entire vectors may be prohibitive, and it may be imperative for them to use sparsification techniques. While most existing work on sparsified mean estimation is agnostic to the characteristics of the data vectors, in many practical applications such as federated learning, there may be spatial correlations (similarities in the vectors sent by different nodes) or temporal correlations (similarities in the data sent by a single node over different iterations of the algorithm) in the data vectors. We leverage these correlations by simply modifying the decoding method used by the server to estimate the mean. We provide an analysis of the resulting estimation error as well as experiments for PCA, K-Means and Logistic Regression, which show that our estimators consistently outperform more sophisticated and expensive sparsification methods.

📄 PDF Abstract BibTeX arXiv:2110.07751

Code (0)

등록된 구현이 없습니다.

Tasks

Federated Learning

Methods 이 논문이 사용한 방법론

PCA Principle Components Analysis (PCA) is an unsupervised method primary used for dimensionality reduction within machine learning. PCA is calculated via a singular value…

Logistic Regression Logistic Regression, despite its name, is a linear model for classification rather than regression. Logistic regression is also known in the literature as logit regression,…

Efficient Covariance Estimation for Sparsified Functional Data

2025-11-23 · Sijie Zheng, Fandong Meng, Jie Zhou arxiv

Motivated by recent work involving the analysis of leveraging spatial correlations in sparsified mean estimation, we present a novel procedure for constructing covariance estimator. The proposed Random-knots (Random-knot…

Spatial-Temporal Synchronous Graph Convolutional Networks: A New Framework for Spatial-Temporal Network Data Forecasting

2020-04-03 · Chao Song, Youfang Lin, Shengnan Guo, Huaiyu Wan

Spatial-temporal network data forecasting is of great importance in a huge amount of applications for traffic management and urban planning. However, the underlying complex spatial-temporal correlations and heterogeneiti…

Management Traffic Prediction

Statistical control for spatio-temporal MEG/EEG source imaging with desparsified multi-task Lasso

2020-09-29 · Jérôme-Alexis Chevalier, Alexandre Gramfort, Joseph Salmon, Bertrand Thirion

Detecting where and when brain regions activate in a cognitive task or in a given clinical condition is the promise of non-invasive techniques like magnetoencephalography (MEG) or electroencephalography (EEG). This probl…

Clustering Constrained Clustering EEG Electroencephalogram (EEG)+2

Statistical control for spatio-temporal MEG/EEG source imaging with desparsified mutli-task Lasso

2020-12-01 · NeurIPS 2020 12 · Jerome-Alexis Chevalier, Joseph Salmon, Alexandre Gramfort, Bertrand Thirion

Clustering Constrained Clustering EEG Electroencephalogram (EEG)+2

Correlation Aware Sparsified Mean Estimation Using Random Projection

2023-10-29 · NeurIPS 2023 11

We study the problem of communication-efficient distributed vector mean estimation, a commonly used subroutine in distributed optimization and Federated Learning (FL). Rand-$k$ sparsification is a commonly used technique…

Distributed Optimization Federated Learning