Reproducibility in Optimization: Theoretical Framework and Limits
We initiate a formal study of reproducibility in optimization. We define a quantitative measure of reproducibility of optimization procedures in the face of noisy or error-prone operations such as inexact or stochastic gradient computations or inexact initialization. We then analyze several convex optimization settings of interest such as smooth, non-smooth, and strongly-convex objective functions and establish tight bounds on the limits of reproducibility in each setting. Our analysis reveals a fundamental trade-off between computation and reproducibility: more computation is necessary (and sufficient) for better reproducibility.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Riddled basin geometry sets fundamental limits to predictability and reproducibility in deep learning
Fundamental limits to predictability are central to our understanding of many physical and computational systems. Here we show that, despite its remarkable capabilities, deep learning exhibits such fundamental limits roo…
Designing Empirical Studies on LLM-Based Code Generation: Towards a Reference Framework
The rise of large language models (LLMs) has introduced transformative potential in automated code generation, addressing a wide range of software engineering challenges. However, empirical evaluation of LLM-based code g…
Code GenerationHeterogeneous Multi-Agent Proximal Policy Optimization for Power Distribution System Restoration
Restoring power distribution systems (PDSs) after large-scale outages requires sequential switching actions that reconfigure feeder topology and coordinate distributed energy resources (DERs) under nonlinear constraints,…
Reinforcement LearningOptimal Guarantees for Algorithmic Reproducibility and Gradient Complexity in Convex Optimization
Algorithmic reproducibility measures the deviation in outputs of machine learning algorithms upon minor changes in the training process. Previous work suggests that first-order methods would need to trade-off convergence…
Sources of Irreproducibility in Machine Learning: A Review
Background: Many published machine learning studies are irreproducible. Issues with methodology and not properly accounting for variation introduced by the algorithm themselves or their implementations are attributed as …
AttributeBIG-bench Machine LearningExperimental Design