Data Virtualization for Machine Learning
Nowadays, machine learning (ML) teams have multiple concurrent ML workflows for different applications. Each workflow typically involves many experiments, iterations, and collaborative activities and commonly takes months and sometimes years from initial data wrangling to model deployment. Organizationally, there is a large amount of intermediate data to be stored, processed, and maintained. \emph{Data virtualization} becomes a critical technology in an infrastructure to serve ML workflows. In this paper, we present the design and implementation of a data virtualization service, focusing on its service architecture and service operations. The infrastructure currently supports six ML applications, each with more than one ML workflow. The data virtualization service allows the number of applications and workflows to grow in the coming years.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Softwarization, Virtualization, & Machine Learning For Intelligent & Effective V2X Communications
The concept of the fifth generation (5G) mobile network system has emerged in recent years as telecommunication operators and service providers look to upgrade their infrastructure and delivery modes to meet the growing …
BIG-bench Machine LearningEdge-computingUnified Distributed Environment
We propose Unified Distributed Environment (UDE), an environment virtualization toolkit for reinforcement learning research. UDE is designed to integrate environments built on any simulation platform such as Gazebo, Unit…
OpenAI Gymreinforcement-learningReinforcement Learning (RL)UnityData Quality Matters: Quantifying Image Quality Impact on Machine Learning Performance
Precise perception of the environment is essential in highly automated driving systems, which rely on machine learning tasks such as object detection and segmentation. Compression of sensor data is commonly used for data…
object-detectionObject DetectionOptimal Virtualization Framework for Cellular Networks with Downlink Rate Coverage Probability Constraints
Wireless network virtualization is emerging as an important technology for next-generation (5G) wireless networks. A key advantage of introducing virtualization in cellular networks is that service providers can robustly…
Stochastic OptimizationHardware-Assisted Virtualization of Neural Processing Units for Cloud Platforms
Cloud platforms today have been deploying hardware accelerators like neural processing units (NPUs) for powering machine learning (ML) inference services. To maximize the resource utilization while ensuring reasonable qu…
Scheduling