Composite Optimization Algorithms for Sigmoid Networks
In this paper, we use composite optimization algorithms to solve sigmoid networks. We equivalently transfer the sigmoid networks to a convex composite optimization and propose the composite optimization algorithms based on the linearized proximal algorithms and the alternating direction method of multipliers. Under the assumptions of the weak sharp minima and the regularity condition, the algorithm is guaranteed to converge to a globally optimal solution of the objective function even in the case of non-convex and non-smooth problems. Furthermore, the convergence results can be directly related to the amount of training data and provide a general guide for setting the size of sigmoid networks. Numerical experiments on Franke's function fitting and handwritten digit recognition show that the proposed algorithms perform satisfactorily and robustly.
Code (0)
등록된 구현이 없습니다.
Tasks
Handwritten Digit RecognitionSimilar Papers 제목 키워드 기반
Zeroth-Order Methods for Stochastic Nonconvex Nonsmooth Composite Optimization
This work aims to solve a stochastic nonconvex nonsmooth composite optimization problem. Previous works on composite optimization problem requires the major part to satisfy Lipschitz smoothness or some relaxed smoothness…
Benchmarking Deep AUROC Optimization: Loss Functions and Algorithmic Choices
The area under the ROC curve (AUROC) has been vigorously applied for imbalanced classification and moreover combined with deep learning techniques. However, there is no existing work that provides sound information for p…
Benchmarkingimbalanced classificationA Composite Activation Function for Learning Stable Binary Representations
Activation functions play a central role in neural networks by shaping internal representations. Recently, learning binary activation representations has attracted significant attention due to their advantages in computa…
Gradient Sliding for Composite Optimization
We consider in this paper a class of composite optimization problems whose objective function is given by the summation of a general smooth and nonsmooth component, together with a relatively simple nonsmooth term. We pr…
Composite Material Design for Optimized Fracture Toughness Using Machine Learning
This paper investigates the optimization of 2D and 3D composite structures using machine learning (ML) techniques, focusing on fracture toughness and crack propagation in the Double Cantilever Beam (DCB) test. By explori…
Cantilever Beam