A generic and fast C++ optimization framework
The development of the mlpack C++ machine learning library (http://www.mlpack.org/) has required the design and implementation of a flexible, robust optimization system that is able to solve the types of arbitrary optimization problems that may arise all throughout machine learning problems. In this paper, we present the generic optimization framework that we have designed for mlpack. A key priority in the design was ease of implementation of both new optimizers and new objective functions to be optimized; therefore, implementation of a new optimizer requires only one method and implementation of a new objective function requires at most four functions. This leads to simple and intuitive code, which, for fast prototyping and experimentation, is of paramount importance. When compared to optimization frameworks of other libraries, we find that mlpack's supports more types of objective functions, is able to make optimizations that other frameworks do not, and seamlessly supports user-defined objective functions and optimizers.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningSimilar Papers 제목 키워드 기반
Faster Margin Maximization Rates for Generic and Adversarially Robust Optimization Methods
First-order optimization methods tend to inherently favor certain solutions over others when minimizing an underdetermined training objective that has multiple global optima. This phenomenon, known as implicit bias, play…
Binary ClassificationGeneric Adaptation by Fast Chaotic Exploration and Slow Feedback Fixation
Living systems adapt to various environmental conditions by changing their internal states. Inspired by gene expression and epigenetic modification dynamics, we herein propose a generic mechanism for adaptation by combin…
FluidML: Fast and Memory Efficient Inference Optimization
Machine learning models deployed on edge devices have enabled numerous exciting new applications, such as humanoid robots, AR glasses, and autonomous vehicles. However, the computing resources available on these edge dev…
Autonomous VehiclesInference OptimizationManagementAdaptive Stochastic Natural Gradient Method for One-Shot Neural Architecture Search
High sensitivity of neural architecture search (NAS) methods against their input such as step-size (i.e., learning rate) and search space prevents practitioners from applying them out-of-the-box to their own problems, al…
image-classificationImage ClassificationNeural Architecture SearchImage Restoration using Plug-and-Play CNN MAP Denoisers
Plug-and-play denoisers can be used to perform generic image restoration tasks independent of the degradation type. These methods build on the fact that the Maximum a Posteriori (MAP) optimization can be solved using sma…
DenoisingImage DenoisingImage Restoration