Papers Distributed Optimization
“Distributed Optimization” 태그가 달린 논문 601편 · 필터 해제
Distributed Online Optimization with Byzantine Adversarial Agents
We study the problem of non-constrained, discrete-time, online distributed optimization in a multi-agent system where some of the agents do not follow the prescribed update rule either due to failures or malicious intent…
Distributed OptimizationToward Communication Efficient Adaptive Gradient Method
In recent years, distributed optimization is proven to be an effective approach to accelerate training of large scale machine learning models such as deep neural networks. With the increasing computation power of GPUs, t…
BIG-bench Machine LearningDistributed OptimizationFederated LearningAsynchronous Iterations in Optimization: New Sequence Results and Sharper Algorithmic Guarantees
We introduce novel convergence results for asynchronous iterations that appear in the analysis of parallel and distributed optimization algorithms. The results are simple to apply and give explicit estimates for how the …
Distributed OptimizationOn the Convergence of Decentralized Adaptive Gradient Methods
Adaptive gradient methods including Adam, AdaGrad, and their variants have been very successful for training deep learning models, such as neural networks. Meanwhile, given the need for distributed computing, distributed…
Distributed ComputingDistributed OptimizationThe Minimax Complexity of Distributed Optimization
In this thesis, I study the minimax oracle complexity of distributed stochastic optimization. First, I present the "graph oracle model", an extension of the classic oracle complexity framework that can be applied to stud…
Distributed OptimizationStochastic OptimizationPrivate Multi-Task Learning: Formulation and Applications to Federated Learning
Many problems in machine learning rely on multi-task learning (MTL), in which the goal is to solve multiple related machine learning tasks simultaneously. MTL is particularly relevant for privacy-sensitive applications i…
BIG-bench Machine LearningDistributed OptimizationFederated LearningMulti-Task Learning+1FL-MISR: Fast Large-Scale Multi-Image Super-Resolution for Computed Tomography Based on Multi-GPU Acceleration
Multi-image super-resolution (MISR) usually outperforms single-image super-resolution (SISR) under a proper inter-image alignment by explicitly exploiting the inter-image correlation. However, the large computational dem…
Computed Tomography (CT)CPUDistributed OptimizationGPU+2Dynamic communication topologies for distributed heuristics in energy system optimization algorithms
The communication topology is an essential aspect in designing distributed optimization heuristics. It can influence the exploration and exploitation of the search space and thus the optimization performance in terms of …
Distributed OptimizationDistributed System Identification for Linear Stochastic Systems with Binary Sensors
The problem of distributed identification of linear stochastic system with unknown coefficients over time-varying networks is considered. For estimating the unknown coefficients, each agent in the network can only access…
Distributed OptimizationLocal SGD Optimizes Overparameterized Neural Networks in Polynomial Time
In this paper we prove that Local (S)GD (or FedAvg) can optimize deep neural networks with Rectified Linear Unit (ReLU) activation function in polynomial time. Despite the established convergence theory of Local SGD on o…
Distributed OptimizationCANITA: Faster Rates for Distributed Convex Optimization with Communication Compression
Due to the high communication cost in distributed and federated learning, methods relying on compressed communication are becoming increasingly popular. Besides, the best theoretically and practically performing gradient…
Distributed OptimizationFederated LearningA Differential Private Method for Distributed Optimization in Directed Networks via State Decomposition
In this paper, we study the problem of consensus-based distributed optimization where a network of agents, abstracted as a directed graph, aims to minimize the sum of all agents' cost functions collaboratively. In existi…
Distributed OptimizationBAGUA: Scaling up Distributed Learning with System Relaxations
Recent years have witnessed a growing list of systems for distributed data-parallel training. Existing systems largely fit into two paradigms, i.e., parameter server and MPI-style collective operations. On the algorithmi…
Distributed OptimizationQuantizationRobust Distributed Optimization With Randomly Corrupted Gradients
In this paper, we propose a first-order distributed optimization algorithm that is provably robust to Byzantine failures-arbitrary and potentially adversarial behavior, where all the participating agents are prone to fai…
Distributed OptimizationStochastic OptimizationAccelerating variational quantum algorithms with multiple quantum processors
Variational quantum algorithms (VQAs) have the potential of utilizing near-term quantum machines to gain certain computational advantages over classical methods. Nevertheless, modern VQAs suffer from cumbersome computati…
Distributed OptimizationAsynchronous Stochastic Optimization Robust to Arbitrary Delays
We consider stochastic optimization with delayed gradients where, at each time step $t$, the algorithm makes an update using a stale stochastic gradient from step $t - d_t$ for some arbitrary delay $d_t$. This setting ab…
Distributed OptimizationStochastic OptimizationSecure Distributed Training at Scale
Many areas of deep learning benefit from using increasingly larger neural networks trained on public data, as is the case for pre-trained models for NLP and computer vision. Training such models requires a lot of computa…
Distributed OptimizationImage ClassificationLanguage ModellingDeepLM: Large-Scale Nonlinear Least Squares on Deep Learning Frameworks Using Stochastic Domain Decomposition
We propose a novel approach for large-scale nonlinear least squares problems based on deep learning frameworks. Nonlinear least squares are commonly solved with the Levenberg-Marquardt (LM) algorithm for fast converg…
Deep LearningDistributed OptimizationLearning Autonomy in Management of Wireless Random Networks
This paper presents a machine learning strategy that tackles a distributed optimization task in a wireless network with an arbitrary number of randomly interconnected nodes. Individual nodes decide their optimal states w…
Distributed OptimizationManagementDecentralized Personalized Federated Learning for Min-Max Problems
Personalized Federated Learning (PFL) has witnessed remarkable advancements, enabling the development of innovative machine learning applications that preserve the privacy of training data. However, existing theoretical …
Distributed OptimizationFederated LearningPersonalized Federated Learning