We need to talk about random seeds
Modern neural network libraries all take as a hyperparameter a random seed, typically used to determine the initial state of the model parameters. This position piece argues that there are some safe uses for random seeds: as part of the hyperparameter search to select a good model, creating an ensemble of several models, or measuring the sensitivity of the training algorithm to the random seed hyperparameter. It argues that some uses for random seeds are risky: using a fixed random seed for ``replicability'' and varying only the random seed to create score distributions for performance comparison. An analysis of 85 recent publications from the ACL Anthology shows that more than 50% contain risky uses of random seeds.
Code (0)
등록된 구현이 없습니다.
Tasks
PositionSimilar Papers 제목 키워드 기반
We need to talk about random seeds
Modern neural network libraries all take as a hyperparameter a random seed, typically used to determine the initial state of the model parameters. This opinion piece argues that there are some safe uses for random seeds:…
Assessing the Macro and Micro Effects of Random Seeds on Fine-Tuning Large Language Models
The impact of random seeds in fine-tuning large language models (LLMs) has been largely overlooked despite its potential influence on model performance.In this study, we systematically evaluate the effects of random seed…
Torch.manual_seed(3407) is all you need: On the influence of random seeds in deep learning architectures for computer vision
In this paper I investigate the effect of random seed selection on the accuracy when using popular deep learning architectures for computer vision. I scan a large amount of seeds (up to $10^4$) on CIFAR 10 and I also sca…
AllThe Power of $D$-hops in Matching Power-Law Graphs
This paper studies seeded graph matching for power-law graphs. Assume that two edge-correlated graphs are independently edge-sampled from a common parent graph with a power-law degree distribution. A set of correctly mat…
Graph MatchingBayesian Optimization Allowing for Common Random Numbers
Bayesian optimization is a powerful tool for expensive stochastic black-box optimization problems such as simulation-based optimization or machine learning hyperparameter tuning. Many stochastic objective functions impli…
Bayesian OptimizationBIG-bench Machine Learning