paper-with-me

홈 › Papers

Distributed Gradient Descent for Functional Learning

2023-05-12 · Zhan Yu, Jun Fan, Zhongjie Shi, Ding-Xuan Zhou

In recent years, different types of distributed and parallel learning schemes have received increasing attention for their strong advantages in handling large-scale data information. In the information era, to face the big data challenges {that} stem from functional data analysis very recently, we propose a novel distributed gradient descent functional learning (DGDFL) algorithm to tackle functional data across numerous local machines (processors) in the framework of reproducing kernel Hilbert space. Based on integral operator approaches, we provide the first theoretical understanding of the DGDFL algorithm in many different aspects of the literature. On the way of understanding DGDFL, firstly, a data-based gradient descent functional learning (GDFL) algorithm associated with a single-machine model is proposed and comprehensively studied. Under mild conditions, confidence-based optimal learning rates of DGDFL are obtained without the saturation boundary on the regularity index suffered in previous works in functional regression. We further provide a semi-supervised DGDFL approach to weaken the restriction on the maximal number of local machines to ensure optimal rates. To our best knowledge, the DGDFL provides the first divide-and-conquer iterative training approach to functional learning based on data samples of intrinsically infinite-dimensional random functions (functional covariates) and enriches the methodologies for functional data analysis.

📄 PDF Abstract BibTeX arXiv:2305.07408

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Natural Gradient for Combined Loss Using Wavelets

2020-06-29 · Lexing Ying

Natural gradients have been widely used in optimization of loss functionals over probability space, with important examples such as Fisher-Rao gradient descent for Kullback-Leibler divergence, Wasserstein gradient descen…

An efficient distributed learning algorithm based on effective local functional approximations

2013-10-31 · Dhruv Mahajan, Nikunj Agrawal, S. Sathiya Keerthi, S. Sundararajan 외

Scalable machine learning over big data is an important problem that is receiving a lot of attention in recent years. On popular distributed environments such as Hadoop running on a cluster of commodity machines, communi…

L2 Regularization

A Mirror Descent-Based Algorithm for Corruption-Tolerant Distributed Gradient Descent

2024-07-19 · Shuche Wang, Vincent Y. F. Tan

Distributed gradient descent algorithms have come to the fore in modern machine learning, especially in parallelizing the handling of large datasets that are distributed across several workers. However, scant attention h…

Distributed Optimization

ErasureHead: Distributed Gradient Descent without Delays Using Approximate Gradient Coding

2019-01-28 · Hongyi Wang, Zachary Charles, Dimitris Papailiopoulos

We present ErasureHead, a new approach for distributed gradient descent (GD) that mitigates system delays by employing approximate gradient coding. Gradient coded distributed GD uses redundancy to exactly recover the gra…

Improving Metric Dimensionality Reduction with Distributed Topology

2021-06-14 · Alexander Wagner, Elchanan Solomon, Paul Bendich

We propose a novel approach to dimensionality reduction combining techniques of metric geometry and distributed persistent homology, in the form of a gradient-descent based method called DIPOLE. DIPOLE is a dimensionalit…

Dimensionality Reduction