Clairvoyant Prefetching for Distributed Machine Learning I/O

2021-01-21 · Nikoli Dryden, Roman Böhringer, Tal Ben-Nun, Torsten Hoefler

I/O is emerging as a major bottleneck for machine learning training, especially in distributed environments. Indeed, at large scale, I/O takes as much as 85% of training time. Addressing this I/O bottleneck necessitates careful optimization, as optimal data ingestion pipelines differ between systems, and require a delicate balance between access to local storage, external filesystems, and remote nodes. We introduce NoPFS, a machine learning I/O middleware, which provides a scalable, flexible, and easy-to-use solution to the I/O bottleneck. NoPFS uses clairvoyance: Given the seed generating the random access pattern for training with SGD, it can exactly predict when and where a sample will be accessed. We combine this with an analysis of access patterns and a performance model to provide distributed caching policies that adapt to different datasets and storage hierarchies. NoPFS reduces I/O times and improves end-to-end training by up to 5.4x on the ImageNet-1k, ImageNet-22k, and CosmoFlow datasets.

📄 PDF Abstract BibTeX arXiv:2101.08734

Code (0)

등록된 구현이 없습니다.

Tasks

BIG-bench Machine Learning

Methods 이 논문이 사용한 방법론

SGD Stochastic Gradient Descent is an iterative optimization technique that uses minibatches of data to form an expectation of the gradient, rather than the full gradient using…

Rudder: Steering Prefetching in Distributed GNN Training using LLM Agents

2026-02-26 · Aishwarya Sarkar, Sayan Ghosh, Nathan Tallent, Aman Chadha 외 arxiv

Large-scale Graph Neural Networks (GNNs) are typically trained by sampling a vertex's neighbors to a fixed distance. Because large input graphs are distributed, training requires frequent irregular communication that sta…

Fine-Grained Address Segmentation for Attention-Based Variable-Degree Prefetching

2022-05-01 · Pengmiao Zhang, Ajitesh Srivastava, Anant V. Nori, Rajgopal Kannan 외

Machine learning algorithms have shown potential to improve prefetching performance by accurately predicting future memory accesses. Existing approaches are based on the modeling of text prediction, considering prefetchi…

Prediction Segmentation

DEAP Cache: Deep Eviction Admission and Prefetching for Cache

2020-09-19 · Ayush Mangal, Jitesh Jain, Keerat Kaur Guliani, Omkar Bhalerao

Recent approaches for learning policies to improve caching, target just one out of the prefetching, admission and eviction processes. In contrast, we propose an end to end pipeline to learn all three policies using machi…

BIG-bench Machine Learning Density Estimation

DeePref: Deep Reinforcement Learning For Video Prefetching In Content Delivery Networks

2023-10-11 · Nawras Alkassab, Chin-Tser Huang, Tania Lorido Botran

Content Delivery Networks carry the majority of Internet traffic, and the increasing demand for video content as a major IP traffic across the Internet highlights the importance of caching and prefetching optimization al…

Deep Reinforcement Learning reinforcement-learning Transfer Learning

A New Design of Cache-aided Multiuser Private Information Retrieval with Uncoded Prefetching

2021-02-02 · Xiang Zhang, Kai Wan, Hua Sun, Mingyue Ji 외

In the problem of cache-aided multiuser private information retrieval (MuPIR), a set of $K_{\rm u}$ cache-equipped users wish to privately download a set of messages from $N$ distributed databases each holding a library …

Information Retrieval Retrieval