An interior-point stochastic approximation method and an L1-regularized delta rule
The stochastic approximation method is behind the solution to many important, actively-studied problems in machine learning. Despite its far-reaching application, there is almost no work on applying stochastic approximation to learning problems with constraints. The reason for this, we hypothesize, is that no robust, widely-applicable stochastic approximation method exists for handling such problems. We propose that interior-point methods are a natural solution. We establish the stability of a stochastic interior-point approximation method both analytically and empirically, and demonstrate its utility by deriving an on-line learning algorithm that also performs feature selection via L1 regularization.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine Learningfeature selectionSimilar Papers 제목 키워드 기반
A Value-Function-based Interior-point Method for Non-convex Bi-level Optimization
Bi-level optimization model is able to capture a wide range of complex learning tasks with practical interest. Due to the witnessed efficiency in solving bi-level programs, gradient-based methods have gained popularity i…
Stochastic Approximation for Risk-aware Markov Decision Processes
We develop a stochastic approximation-type algorithm to solve finite state/action, infinite-horizon, risk-aware Markov decision processes. Our algorithm has two loops. The inner loop computes the risk by solving a stocha…
Q-LearningOptimized Energy Harvesting in Cell-Free Massive MIMO Using Markov Process Evolution
This paper investigates a discrete energy state transition model for energy harvesting (EH) in cell-free massive multiple-input-multiple-output (CF-mMIMO) networks. A Markov chain-based stochastic process is conceived to…
Single-Loop Deterministic and Stochastic Interior-Point Algorithms for Nonlinearly Constrained Optimization
An interior-point algorithm framework is proposed, analyzed, and tested for solving nonlinearly constrained continuous optimization problems. The main setting of interest is when the objective and constraint functions ma…
Unregularized limit of stochastic gradient method for Wasserstein distributionally robust optimization
Distributionally robust optimization offers a compelling framework for model fitting in machine learning, as it systematically accounts for data uncertainty. Focusing on Wasserstein distributionally robust optimization, …