ROMUL: Scale Adaptative Population Based Training
In most pragmatic settings, data augmentation and regularization are essential, and require hyperparameter search. Population based training (PBT) is an effective tool for efficiently finding them as well as schedules over hyperparameters. In this paper, we compare existing PBT algorithms and contribute a new one: ROMUL, for RObust MULtistep search, which adapts its stepsize over the course of training. We report competitive results with standard models on CIFAR (image classification) as well as Penn Tree Bank (language modeling), which both depend on heavy regularization. We also open-source hoptim, a PBT library agnostic to the training framework, which is simple to use, reentrant, and provides good defaults with ROMUL.
Code (0)
등록된 구현이 없습니다.
Tasks
Data Augmentationimage-classificationImage ClassificationLanguage ModelingLanguage ModellingSimilar Papers 제목 키워드 기반
A Bayesian Variational principle for dynamic Self Organizing Maps
We propose organisation conditions that yield a method for training SOM with adaptative neighborhood radius in a variational Bayesian framework. This method is validated on a non-stationary setting and compared in an hig…
Benchmarking Adaptative Variational Quantum Algorithms on QUBO Instances
In recent years, Variational Quantum Algorithms (VQAs) have emerged as a promising approach for solving optimization problems on quantum computers in the NISQ era. However, one limitation of VQAs is their reliance on fix…
BenchmarkingLELIO: An Auto-Adaptative System to Acquire Domain Lexical Knowledge in Technical Texts
In this paper, we investigate some language acquisition facets of an auto-adaptative system that can automatically acquire most of the relevant lexical knowledge and authoring practices for an application in a given doma…
Language AcquisitionAn adaptative differential evolution with enhanced diversity and restart mechanism
Differential Evolution (DE) stands out as an exceptional intelligent evolutionary algorithm, acclaimed for its simplicity in implementation and the ability to optimize without necessitating differentiable conditions. How…
DiversityAGAR: Attention Graph-RNN for Adaptative Motion Prediction of Point Clouds of Deformable Objects
This paper focuses on motion prediction for point cloud sequences in the challenging case of deformable 3D objects, such as human body motion. First, we investigate the challenges caused by deformable shapes and complex …
Action Recognitionmotion prediction