DARWIN: Dynamic Agentically Rewriting Self-Improving Network
DARWIN is an evolutionary GPT model, utilizing a genetic-algorithm like optimization structure with several independent GPT agents being trained individually using unique training code. Each iteration, the GPT models are prompted to modify the training code of one another in an attempt to improve their performance in a mutation-like manner, and the best GPT agents are then benchmarked and selected for the next iteration by genetic algorithm. For demonstration purposes and due to budget and time constraints, OpenAI API is used to prompt training code improvements and the nanoGPT framework is used as the training code. DARWIN also utilizes persistent JSON-based memory files to track previous reasoning and changes to code to correlate with improvement to model performance. and a bidirectional interface for HITL intervention allowing the model to request upgrades such as additional datasets, training scripts, and restructuring of file hierarchies. In experiments, DARWIN achieved a 1.26 percent improvement in model FLOPS utilization (MFU) and a 2.07 percent improvement to perplexity in 5 iterations of training over baseline configurations, demonstrating promising capabilities as a foundation for scaling evolutionary GPT training.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Darwinian Evolution of Taste
What is life. Schrodingers question is discussed here for a specific protein, villin, which builds cells in tissues that detect taste and sound. Villin is represented by a sequence of 827 amino acids bound to a peptide b…
Self-Organized Networks: Darwinian Evolution of Myosin-1
Cytoskeletons are self-organized networks based on polymerized proteins: actin, tubulin, and driven by motor proteins, such as myosin, kinesin and dynein. Their positive Darwinian evolution enables them to approach optim…
Self-Organized Networks, Darwinian Evolution of Self-Organized Networks, Darwinian Evolution of Dynein Rings, Stalks and Stalk Heads
Cytoskeletons are self organized networks based on polymerized proteins, actin, tubulin, and driven by motor proteins, such as myosin, kinesin and dynein. Their positive Darwinian evolution enables them to approach optim…
Self-Organized Networks with Long-Range Interactions: Tandem Darwinian Evolution of $α$ and $β$ Tubulin
Cytoskeletons are self-organized networks based on polymerized proteins: actin, tubulin, and driven by motor proteins, such as myosin, kinesin and dynein. Their positive Darwinian evolution enables them to approach optim…
How Life Works: Darwinian Evolution of Proteins
We review the development of thermodynamic protein hydropathic scaling theory, starting from backgrounds in mathematics and statistical mechanics, and leading to biomedical applications. Darwinian evolution has organized…
Ingenuity