Distributed Double Machine Learning with a Serverless Architecture
This paper explores serverless cloud computing for double machine learning. Being based on repeated cross-fitting, double machine learning is particularly well suited to exploit the high level of parallelism achievable with serverless computing. It allows to get fast on-demand estimations without additional cloud maintenance effort. We provide a prototype Python implementation \texttt{DoubleML-Serverless} for the estimation of double machine learning models with the serverless computing platform AWS Lambda and demonstrate its utility with a case study analyzing estimation times and costs.
Code (1)
Tasks
BIG-bench Machine LearningCloud ComputingSimilar Papers 제목 키워드 기반
Architecting Peer-to-Peer Serverless Distributed Machine Learning Training for Improved Fault Tolerance
Distributed Machine Learning refers to the practice of training a model on multiple computers or devices that can be called nodes. Additionally, serverless computing is a new paradigm for cloud computing that uses functi…
Cloud ComputingDecision MakingSPIRT: A Fault-Tolerant and Reliable Peer-to-Peer Serverless ML Training Architecture
The advent of serverless computing has ushered in notable advancements in distributed machine learning, particularly within parameter server-based architectures. Yet, the integration of serverless features within peer-to…
Exploring the Impact of Serverless Computing on Peer To Peer Training Machine Learning
The increasing demand for computational power in big data and machine learning has driven the development of distributed training methodologies. Among these, peer-to-peer (P2P) networks provide advantages such as enhance…
Optimizing Distributed Deployment of Mixture-of-Experts Model Inference in Serverless Computing
With the advancement of serverless computing, running machine learning (ML) inference services over a serverless platform has been advocated, given its labor-free scalability and cost effectiveness. Mixture-of-Experts (M…
Bayesian OptimizationCPUMixture-of-ExpertsFSD-Inference: Fully Serverless Distributed Inference with Scalable Cloud Communication
Serverless computing offers attractive scalability, elasticity and cost-effectiveness. However, constraints on memory, CPU and function runtime have hindered its adoption for data-intensive applications and machine learn…
CPU