Kareus: Joint Reduction of Dynamic and Static Energy in Large Model Training
The computing demand of AI is growing at an unprecedented rate, but energy supply is not keeping pace. As a result, energy has become an expensive and contended resource that requires explicit management and optimization. Although recent works have made significant progress in large model training optimization, they focus on optimizing either dynamic or static energy consumption. We find that fine-grained kernel scheduling and frequency scaling jointly and interdependently impact both dynamic and static energy consumption. Based on this finding, we design Kareus, a training system that pushes the time-energy tradeoff frontier by optimizing both aspects. Kareus decomposes the intractable joint optimization problem into local, partition-based subproblems. It then uses a multi-pass multi-objective optimization algorithm to find execution schedules that push the time-energy tradeoff frontier. Compared to the state of the art, Kareus reduces training energy by up to 28.3% at the same training time, or reduces training time by up to 27.5% at the same energy consumption.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Dynamic Decision Tree Ensembles for Energy-Efficient Inference on IoT Edge Nodes
With the increasing popularity of Internet of Things (IoT) devices, there is a growing need for energy-efficient Machine Learning (ML) models that can run on constrained edge nodes. Decision tree ensembles, such as Rando…
C++ codeS2TA: Exploiting Structured Sparsity for Energy-Efficient Mobile CNN Acceleration
Exploiting sparsity is a key technique in accelerating quantized convolutional neural network (CNN) inference on mobile devices. Prior sparse CNN accelerators largely exploit un-structured sparsity and achieve significan…
A 97 fJ/Conversion Neuron-ADC with Reconfigurable Sampling and Static Power Reduction
A bio-inspired Neuron-ADC with reconfigurable sampling and static power reduction for biomedical applications is proposed in this work. The Neuron-ADC leverages level-crossing sampling and a bio-inspired refractory circu…
Revisiting Dynamic Convolution via Matrix Decomposition
Recent research in dynamic convolution shows substantial performance boost for efficient CNNs, due to the adaptive aggregation of K static convolution kernels. It has two limitations: (a) it increases the number of convo…
Dimensionality ReductionA Dynamic Equivalent Energy Storage Model of Natural Gas Networks for Joint Optimal Dispatch of Electricity-Gas Systems
The development of energy conversion techniques enhances the coupling between the gas network and power system. However, challenges remain in the joint optimal dispatch of electricity-gas systems. The dynamic model of th…