Transactional Python for Durable Machine Learning: Vision, Challenges, and Feasibility
In machine learning (ML), Python serves as a convenient abstraction for working with key libraries such as PyTorch, scikit-learn, and others. Unlike DBMS, however, Python applications may lose important data, such as trained models and extracted features, due to machine failures or human errors, leading to a waste of time and resources. Specifically, they lack four essential properties that could make ML more reliable and user-friendly -- durability, atomicity, replicability, and time-versioning (DART). This paper presents our vision of Transactional Python that provides DART without any code modifications to user programs or the Python kernel, by non-intrusively monitoring application states at the object level and determining a minimal amount of information sufficient to reconstruct a whole application. Our evaluation of a proof-of-concept implementation with public PyTorch and scikit-learn applications shows that DART can be offered with overheads ranging 1.5%--15.6%.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Agentic Transaction: Towards ACID-Compliant Agent Systems
Large language model (LLM) agents are evolving from conversational assistants into autonomous systems that execute long-horizon tasks through reasoning, tool use, code generation, and workspace manipulation. As agents in…
Code GenerationResilient Write: A Six-Layer Durable Write Surface for LLM Coding Agents
LLM-powered coding agents increasingly rely on tool-use protocols such as the Model Context Protocol (MCP) to read and write files on a developer's workstation. When a write fails - due to content filters, truncation, or…
Fishing for Phishers: Learning-Based Phishing Detection in Ethereum Transactions
Phishing detection on Ethereum has increasingly leveraged advanced machine learning techniques to identify fraudulent transactions. However, limited attention has been given to understanding the effectiveness of feature …
feature selectionHarnessing Scalable Transactional Stream Processing for Managing Large Language Models [Vision]
Large Language Models (LLMs) have demonstrated extraordinary performance across a broad array of applications, from traditional language processing tasks to interpreting structured sequences like time-series data. Yet, t…
Decision MakingManagementTime SeriesTowards WARSHIP: Combining Components of Brain-Inspired Computing of RSH for Image Super Resolution
Evolution of deep learning shows that some algorithmic tricks are more durable , while others are not. To the best of our knowledge, we firstly summarize 5 more durable and complete deep learning components for vision, t…
Deep LearningImage Super-ResolutionSuper-Resolution