RIBBON: Cost-Effective and QoS-Aware Deep Learning Model Inference using a Diverse Pool of Cloud Computing Instances
Deep learning model inference is a key service in many businesses and scientific discovery processes. This paper introduces RIBBON, a novel deep learning inference serving system that meets two competing objectives: quality-of-service (QoS) target and cost-effectiveness. The key idea behind RIBBON is to intelligently employ a diverse set of cloud computing instances (heterogeneous instances) to meet the QoS target and maximize cost savings. RIBBON devises a Bayesian Optimization-driven strategy that helps users build the optimal set of heterogeneous instances for their model inference service needs on cloud computing platforms -- and, RIBBON demonstrates its superiority over existing approaches of inference serving systems using homogeneous instance pools. RIBBON saves up to 16% of the inference service cost for different learning models including emerging deep learning recommender system models and drug-discovery enabling models.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian OptimizationCloud ComputingDeep LearningDrug DiscoveryRecommendation Systemsscientific discoveryMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Approximate Bayesian Inference for a Mechanistic Model of Vesicle Release at a Ribbon Synapse
The inherent noise of neural systems makes it difficult to construct models which accurately capture experimental measurements of their activity. While much research has been done on how to efficiently model neural activ…
Bayesian InferenceDescriptiveRibbon: Scalable Approximation and Robust Uncertainty Quantification
Reliably quantifying predictive uncertainty is difficult for complex, high-dimensional, or misspecified models. Both fully Bayesian and bootstrap resampling methods provide principled uncertainty estimates but are often …
Discrete Elastic Ribbons: A Unified Discrete Differential Geometry Framework for One-Dimensional Energy Models
Elastic ribbons, slender structures whose length ($L$), width ($W$), and thickness ($b$) satisfy $L \gg W \gg b$, exhibit mechanical behaviors intermediate between one-dimensional rods ($L \gg W, b$) and two-dimensional …
Searching for ribbons with machine learning
We apply Bayesian optimization and reinforcement learning to a problem in topology: the question of when a knot bounds a ribbon disk. This question is relevant in an approach to disproving the four-dimensional smooth Poi…
Bayesian Optimizationreinforcement-learningReinforcement LearningPerceptual Robust Hashing for Color Images with Canonical Correlation Analysis
In this paper, a novel perceptual image hashing scheme for color images is proposed based on ring-ribbon quadtree and color vector angle. First, original image is subjected to normalization and Gaussian low-pass filterin…
Copy Detection