Training Overhead Ratio: A Practical Reliability Metric for Large Language Model Training Systems
Large Language Models (LLMs) are revolutionizing the AI industry with their superior capabilities. Training these models requires large-scale GPU clusters and significant computing time, leading to frequent failures that significantly increase training costs. Despite its significance, this field lacks a metric for evaluating reliability. In this work, we introduce a novel reliability metric called \emph{Training Overhead Ratio} (TOR) to evaluate the reliability of fault-tolerant LLM training systems. TOR is defined as the ratio of optimal training time to the observed training time of a system, serving as a practical tool for users to estimate the actual time required to train an LLM on a given system. Furthermore, our investigation identifies the key factor for enhancing reliability and present TOR equations for various types of failures encountered in practice.
Code (0)
등록된 구현이 없습니다.
Tasks
GPULanguage ModelingLanguage ModellingLarge Language ModelSimilar Papers 제목 키워드 기반
Waste Not, Want Not; Recycled Gumbel Noise Improves Consistency in Natural Language Generation
Consistency in the output of language models is critical for their reliability and practical utility. Due to their training objective, language models learn to model the full space of possible continuations, leading to o…
Language ModelingLanguage ModellingText GenerationDiffusion Probe: Generated Image Result Prediction Using CNN Probes
Text-to-image (T2I) diffusion models lack an efficient mechanism for early quality assessment, leading to costly trial-and-error in multi-generation scenarios such as prompt iteration, agent-based generation, and flow-gr…
What is Your Metric Telling You? Evaluating Classifier Calibration under Context-Specific Definitions of Reliability
Classifier calibration has received recent attention from the machine learning community due both to its practical utility in facilitating decision making, as well as the observation that modern neural network classifier…
Classifier calibrationDecision MakingTexture- and Shape-based Adversarial Attacks for Vehicle Detection in Synthetic Overhead Imagery
Detecting vehicles in aerial images can be very challenging due to complex backgrounds, small resolution, shadows, and occlusions. Despite the effectiveness of SOTA detectors such as YOLO, they remain vulnerable to adver…
vehicle detectionFlipcy: Efficient Pattern Redistribution for Enhancing MLC PCM Reliability and Storage Density
Phase change memory (PCM) is a scalable, nonvolatile emerging memory. The storage density of the PCM can be enhanced by using the multi-level cell (MLC) operation. However, the MLC PCM suffers from low reliability due …