DiscoGen: Procedural Generation of Algorithm Discovery Tasks in Machine Learning
Automating the development of machine learning algorithms has the potential to unlock new breakthroughs. However, our ability to improve and evaluate algorithm discovery systems has thus far been limited by existing task suites. They suffer from many issues, such as: poor evaluation methodologies; data contamination; and containing saturated or very similar problems. Here, we introduce DiscoGen, a procedural generator of algorithm discovery tasks for machine learning, such as developing optimisers for reinforcement learning or loss functions for image classification. Motivated by the success of procedural generation in reinforcement learning, DiscoGen spans billions of tasks of varying difficulty and complexity from a range of machine learning fields. These tasks are specified by a small number of configuration parameters and can be used to optimise algorithm discovery agents (ADAs). We present DiscoBench, a fixed, small subset of DiscoGen tasks for principled evaluation of ADAs. Finally, we propose a number of ambitious, impactful research directions enabled by DiscoGen, and demonstrate its use for ADA optimisation through scaling experiments for automated prompt tuning. DiscoGen is released open-source at https://github.com/AlexGoldie/discogen.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement LearningImage ClassificationSimilar Papers 제목 키워드 기반
DiscoGen: Learning to Discover Gene Regulatory Networks
Accurately inferring Gene Regulatory Networks (GRNs) is a critical and challenging task in biology. GRNs model the activatory and inhibitory interactions between genes and are inherently causal in nature. To accurately i…
Causal DiscoveryProcedural Content Metageneration via Program Search and Continual Abstraction Discovery
Large language models can generate executable programs, which makes it possible to search directly over procedural content generators rather than individual levels. We study this approach in Sokoban, Zelda, Dangerous Dav…
Learning to Generate Levels From Nothing
Machine learning for procedural content generation has recently become an active area of research. Levels vary in both form and function and are mostly unrelated to each other across games. This has made it difficult to …
Image GenerationInductive LearningReinforcement LearningPairing Analogy-Augmented Generation with Procedural Memory for Procedural Q&A
Large language models struggle to synthesize disparate pieces of information into a coherent plan when approaching a complex procedural task. In this work, we introduce a novel formalism and structure for such procedural…
Language ModellingQuestion AnsweringRAGGuided Proceduralization: Optimizing Geometry Processing and Grammar Extraction for Architectural Models
We describe a guided proceduralization framework that optimizes geometry processing on architectural input models to extract target grammars. We aim to provide efficient artistic workflows by creating procedural represen…