Pay-Per-Request Deployment of Neural Network Models Using Serverless Architectures
We demonstrate the serverless deployment of neural networks for model inferencing in NLP applications using Amazon{'}s Lambda service for feedforward evaluation and DynamoDB for storing word embeddings. Our architecture realizes a pay-per-request pricing model, requiring zero ongoing costs for maintaining server instances. All virtual machine management is handled behind the scenes by the cloud provider without any direct developer intervention. We describe a number of techniques that allow efficient use of serverless resources, and evaluations confirm that our design is both scalable and inexpensive.
Code (0)
등록된 구현이 없습니다.
Tasks
Answer SelectionManagementQuestion AnsweringSentence ClassificationWord EmbeddingsSimilar Papers 제목 키워드 기반
Performance Modeling of Metric-Based Serverless Computing Platforms
Analytical performance models are very effective in ensuring the quality of service and cost of service deployment remain desirable under different conditions and workloads. While various analytical performance models ha…
Cloud ComputingStraightLine: An End-to-End Resource-Aware Scheduler for Machine Learning Application Requests
The life cycle of machine learning (ML) applications consists of two stages: model development and model deployment. However, traditional ML systems (e.g., training-specific or inference-specific systems) focus on one pa…
DeF-DReL: Systematic Deployment of Serverless Functions in Fog and Cloud environments using Deep Reinforcement Learning
Fog computing is introduced by shifting cloud resources towards the users' proximity to mitigate the limitations possessed by cloud computing. Fog environment made its limited resource available to a large number of user…
Cloud ComputingDeep Reinforcement LearningOptimizing Distributed Deployment of Mixture-of-Experts Model Inference in Serverless Computing
With the advancement of serverless computing, running machine learning (ML) inference services over a serverless platform has been advocated, given its labor-free scalability and cost effectiveness. Mixture-of-Experts (M…
Bayesian OptimizationCPUMixture-of-ExpertsMLProxy: SLA-Aware Reverse Proxy for Machine Learning Inference Serving on Serverless Computing Platforms
Serving machine learning inference workloads on the cloud is still a challenging task on the production level. Optimal configuration of the inference workload to meet SLA requirements while optimizing the infrastructure …
BIG-bench Machine LearningManagement