Fast Training Method for Stochastic Compositional Optimization Problems
The stochastic compositional optimization problem covers a wide range of machine learning models, such as sparse additive models and model-agnostic meta-learning. Thus, it is necessary to develop efficient methods for its optimization. Existing methods for the stochastic compositional optimization problem only focus on the single machine scenario, which is far from satisfactory when data are distributed on different devices. To address this problem, we propose novel decentralized stochastic compositional gradient descent methods to efficiently train the large-scale stochastic compositional optimization problem. To the best of our knowledge, our work is the first one facilitating decentralized training for this kind of problem. Furthermore, we provide the convergence analysis for our methods, which shows that the convergence rate of our methods can achieve linear speedup with respect to the number of devices. At last, we apply our decentralized training methods to the model-agnostic meta-learning problem, and the experimental results confirm the superior performance of our methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Additive modelsMeta-LearningSimilar Papers 제목 키워드 기반
Fast Training Method for Stochastic Compositional Optimization Problems
The stochastic compositional optimization problem covers a wide range of machine learning models, such as sparse additive models and model-agnostic meta-learning. Thus, it is necessary to develop efficient methods for i…
Additive modelsMeta-LearningCompositional Stochastic Average Gradient for Machine Learning and Related Applications
Many machine learning, statistical inference, and portfolio optimization problems require minimization of a composition of expected value functions (CEVF). Of particular interest is the finite-sum versions of such compos…
BIG-bench Machine LearningPortfolio OptimizationEfficient Smooth Non-Convex Stochastic Compositional Optimization via Stochastic Recursive Gradient Descent
Stochastic compositional optimization arises in many important machine learning tasks such as reinforcement learning and portfolio management. The objective function is the composition of two expectations of stochastic f…
ManagementReinforcement LearningStochastic OptimizationStochastic Recursive Variance Reduction for Efficient Smooth Non-Convex Compositional Optimization
Stochastic compositional optimization arises in many important machine learning tasks such as value function evaluation in reinforcement learning and portfolio management. The objective function is the composition of two…
ManagementReinforcement LearningStochastic OptimizationStability and Generalization of Stochastic Compositional Gradient Descent Algorithms
Many machine learning tasks can be formulated as a stochastic compositional optimization (SCO) problem such as reinforcement learning, AUC maximization, and meta-learning, where the objective function involves a nested c…
Learning TheoryMeta-Learning