Hyperparameter Learning via Distributional Transfer

2018-10-15 · NeurIPS 2019 12 · Ho Chung Leon Law, Peilin Zhao, Lucian Chan, Junzhou Huang, Dino Sejdinovic

Bayesian optimisation is a popular technique for hyperparameter learning but typically requires initial exploration even in cases where similar prior tasks have been solved. We propose to transfer information across tasks using learnt representations of training datasets used in those tasks. This results in a joint Gaussian process model on hyperparameters and data representations. Representations make use of the framework of distribution embeddings into reproducing kernel Hilbert spaces. The developed method has a faster convergence compared to existing baselines, in some cases requiring only a few evaluations of the target objective.

📄 PDF Abstract BibTeX arXiv:1810.06305

Code (1)

hcllaw/distBO tf

Tasks

Bayesian Optimisation

Methods 이 논문이 사용한 방법론

Gaussian Process Gaussian Processes are non-parametric models for approximating functions. They rely upon a measure of similarity between points (the kernel function) to predict the value for…

Improving Distributional Similarity with Lessons Learned from Word Embeddings

2015-01-01 · TACL 2015 1 · Omer Levy, Yoav Goldberg, Ido Dagan

Recent trends suggest that neural-network-inspired word embedding models outperform traditional count-based distributional models on word similarity and analogy detection tasks. We reveal that much of the performance gai…

Word Embeddings Word Similarity

When Hyperparameters Help: Beneficial Parameter Combinations in Distributional Semantic Models

2016-08-01 · SEMEVAL 2016 8 · Alicia Krebs, Denis Paperno

Hyperparameter Optimization Word Embeddings Word Similarity

Limits of Model Selection under Transfer Learning

2023-04-29 · Steve Hanneke, Samory Kpotufe, Yasaman Mahdaviyeh

Theoretical studies on transfer learning or domain adaptation have so far focused on situations with a known hypothesis class or model; however in practice, some amount of model selection is usually involved, often appea…

Domain Adaptation model Model Selection Transfer Learning

Completed Hyperparameter Transfer across Modules, Width, Depth, Batch and Duration

2025-12-26 · Bruno Mlodozeniec, Pierre Ablin, Louis Béthune, Dan Busbridge 외 arxiv

Hyperparameter tuning can dramatically impact training stability and final performance of large-scale models. Recent works on neural network parameterisations, such as $μ$P, have enabled transfer of optimal global hyperp…

Revisiting Gaussian mixture critics in off-policy reinforcement learning: a sample-based approach

2022-04-21 · Bobak Shahriari, Abbas Abdolmaleki, Arunkumar Byravan, Abe Friesen 외

Actor-critic algorithms that make use of distributional policy evaluation have frequently been shown to outperform their non-distributional counterparts on many challenging control tasks. Examples of this behavior includ…

continuous-control Continuous Control reinforcement-learning Reinforcement Learning (RL)