InferF: Declarative Factorization of AI/ML Inferences over Joins
Real-world AI/ML workflows often apply inference computations to feature vectors joined from multiple datasets. To avoid the redundant AI/ML computations caused by repeated data records in the join's output, factorized ML has been proposed to decompose ML computations into sub-computations to be executed on each normalized dataset. However, there is insufficient discussion on how factorized ML could impact AI/ML inference over multi-way joins. To address the limitations, we propose a novel declarative InferF system, focusing on the factorization of arbitrary inference workflows represented as analyzable expressions over the multi-way joins. We formalize our problem to flexibly push down partial factorized computations to qualified nodes in the join tree to minimize the overall inference computation and join costs and propose two algorithms to resolve the problem: (1) a greedy algorithm based on a per-node cost function that estimates the influence on overall latency if a subset of factorized computations is pushed to a node, and (2) a genetic algorithm for iteratively enumerating and evaluating promising factorization plans. We implement InferF on Velox, an open-sourced database engine from Meta, evaluate it on real-world datasets, observed up to 11.3x speedups, and systematically summarized the factors that determine when factorized ML can benefit AI/ML inference workflows.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Inferflow: an Efficient and Highly Configurable Inference Engine for Large Language Models
We present Inferflow, an efficient and highly configurable inference engine for large language models (LLMs). With Inferflow, users can serve most of the common transformer models by simply modifying some lines in corres…
GPUQuantizationDeclarative Privacy-Preserving Inference Queries
Detecting inference queries running over personal attributes and protecting such queries from leaking individual information requires tremendous effort from practitioners. To tackle this problem, we propose an end-to-end…
Federated LearningManagementPrivacy PreservingHardware-conscious Hash-Joins on GPUs
Traditionally, analytical database engines have used task parallelism provided by modern multi-socket multicore CPUs for scaling query execution. Over the past few years, GPUs have started gaining traction as accelerator…
CPUGPUBrainSurgery: Reproducible and Reliable Declarative Weight Manipulations for Model Editing and Upcycling
As deep learning models scale, managing, inspecting, and modifying large checkpoints has become increasingly challenging. Researchers often need to alter model weights for layer restructuring, precision casting, low-rank…
Ilargi: a GPU Compatible Factorized ML Model Training Framework
The machine learning (ML) training over disparate data sources traditionally involves materialization, which can impose substantial time and space overhead due to data movement and replication. Factorized learning, which…
Computational EfficiencyCPUData IntegrationGPU