Bauplan: zero-copy, scale-up FaaS for data pipelines
Chaining functions for longer workloads is a key use case for FaaS platforms in data applications. However, modern data pipelines differ significantly from typical serverless use cases (e.g., webhooks and microservices); this makes it difficult to retrofit existing pipeline frameworks due to structural constraints. In this paper, we describe these limitations in detail and introduce bauplan, a novel FaaS programming model and serverless runtime designed for data practitioners. bauplan enables users to declaratively define functional Directed Acyclic Graphs (DAGs) along with their runtime environments, which are then efficiently executed on cloud-based workers. We show that bauplan achieves both better performance and a superior developer experience for data workloads by making the trade-off of reducing generality in favor of data-awareness
Code (1)
Similar Papers 제목 키워드 기반
FaaSMoE: A Serverless Framework for Multi-Tenant Mixture-of-Experts Serving
Mixture-of-Experts (MoE) models offer high capacity with efficient inference cost by activating a small subset of expert models per input. However, deploying MoE models requires all experts to reside in memory, creating …
Prediction of amino acid content in live black soldier fly larvae using near infrared spectroscopy
Black soldier fly (Hermetia illucens) larvae are emerging as a sustainable protein source for animal feed and human nutrition. Ensuring consistent amino acid composition is crucial for quality control, necessitating rapi…
NutritionVerifiable Fairness: Privacy-preserving Computation of Fairness for Machine Learning Systems
Fair machine learning is a thriving and vibrant research topic. In this paper, we propose Fairness as a Service (FaaS), a secure, verifiable and privacy-preserving protocol to computes and verify the fairness of any mach…
FairnessPrivacy PreservingArchitectural Implications of Function-as-a-Service Computing
Serverless computing is a rapidly growing cloud application model, popularized by Amazon’s Lambda platform. Serverless cloud services provide fine-grained provisioning of resources, which scale automatically with user de…
A Framework for SLO, Carbon, and Wastewater-Aware Sustainable FaaS Cloud Platform Management
Function-as-a-Service (FaaS) is a growing cloud computing paradigm that is expected to reduce the user cost of service over traditional serverful approaches. However, the environmental impact of FaaS has not received muc…
Cloud ComputingManagementScheduling