Saturn: Efficient Multi-Large-Model Deep Learning
In this paper, we propose Saturn, a new data system to improve the efficiency of multi-large-model training (e.g., during model selection/hyperparameter optimization). We first identify three key interconnected systems challenges for users building large models in this setting -- parallelism technique selection, distribution of GPUs over jobs, and scheduling. We then formalize these as a joint problem, and build a new system architecture to tackle these challenges simultaneously. Our evaluations show that our joint-optimization approach yields 39-49% lower model selection runtimes than typical current DL practice.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningHyperparameter OptimizationmodelModel SelectionSchedulingSimilar Papers 제목 키워드 기반
SATURN: SAT-based Reinforcement Learning to Unleash Language Model Reasoning
How to design reinforcement learning (RL) tasks that effectively unleash the reasoning capability of large language models (LLMs) remains an open question. Existing RL tasks (e.g., math, programming, and constructing rea…
Language ModelingLanguage ModellingMathReinforcement Learning (RL)SATURN: Symbolic Spatial Reasoning for Multi-Perspective Grounding
Vision-Language Models (VLMs) remain unreliable when spatial reasoning requires composing relations whose meanings depend on frames of reference. Existing neuro-symbolic methods make reasoning more explicit, but often de…
Referring ExpressionSpatial ReasoningSATURN: Autoregressive Image Generation Guided by Scene Graphs
State-of-the-art text-to-image models excel at photorealistic rendering but often struggle to capture the layout and object relationships implied by complex prompts. Scene graphs provide a natural structural prior, yet p…
Image GenerationSaturn Platform: Foundation Model Operations and Generative AI for Financial Services
Saturn is an innovative platform that assists Foundation Model (FM) building and its integration with IT operations (Ops). It is custom-made to meet the requirements of data scientists, enabling them to effectively creat…
Saturn: Sample-efficient Generative Molecular Design using Memory Manipulation
Generative molecular design for drug discovery has very recently achieved a wave of experimental validation, with language-based backbones being the most common architectures employed. The most important factor for downs…
Data AugmentationDrug DiscoveryMamba