Distributed Online Learning with Multiple Kernels
We consider the problem of learning a nonlinear function over a network of learners in a fully decentralized fashion. Online learning is additionally assumed, where every learner receives continuous streaming data locally. This learning model is called a fully distributed online learning (or a fully decentralized online federated learning). For this model, we propose a novel learning framework with multiple kernels, which is named DOMKL. The proposed DOMKL is devised by harnessing the principles of an online alternating direction method of multipliers and a distributed Hedge algorithm. We theoretically prove that DOMKL over T time slots can achieve an optimal sublinear regret, implying that every learner in the network can learn a common function which has a diminishing gap from the best function in hindsight. Our analysis also reveals that DOMKL yields the same asymptotic performance of the state-of-the-art centralized approach while keeping local data at edge learners. Via numerical tests with real datasets, we demonstrate the effectiveness of the proposed DOMKL on various online regression and time-series prediction tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningTime SeriesTime Series AnalysisTime Series PredictionSimilar Papers 제목 키워드 기반
Distributed Online Learning with Multiple Kernels
In the Internet-of-Things (IoT) systems, there are plenty of informative data provided by a massive number of IoT devices (e.g., sensors). Learning a function from such data is of great interest in machine learning tasks…
Privacy PreservingTime SeriesTime Series AnalysisTime Series PredictionPersonalized Online Federated Learning with Multiple Kernels
Multi-kernel learning (MKL) exhibits well-documented performance in online non-linear function approximation. Federated learning enables a group of learners (called clients) to train an MKL model on the data distributed …
Federated LearningDistributed Adaptive Learning with Multiple Kernels in Diffusion Networks
We propose an adaptive scheme for distributed learning of nonlinear functions by a network of nodes. The proposed algorithm consists of a local adaptation stage utilizing multiple kernels with projections onto hyperslabs…
Stabilization of uncertain linear distributed delay systems with dissipativity constraints
This paper examines the problem of stabilizing linear distributed delay systems with nonlinear distributed delay kernels and dissipativity constraints. Specifically, the nonlinear distributed kernel includes functions su…
Manifold Adaptive Multiple Kernel K-Means for Clustering
Multiple kernel methods based on k-means aims to integrate a group of kernels to improve the performance of kernel k-means clustering. However, we observe that most existing multiple kernel k-means methods exploit the no…
Clustering