paper-with-me

Papers

Distributed storage algorithms with optimal tradeoffs

2021-01-13 · Michael Luby, Thomas Richardson

One of the primary objectives of a distributed storage system is to reliably store large amounts of source data for long durations using a large number $N$ of unreliable storage nodes, each with $c$ bits of storage capacity. Storage nodes fail randomly over time and are replaced with nodes of equal capacity initialized to zeroes, and thus bits are erased at some rate $e$. To maintain recoverability of the source data, a repairer continually reads data over a network from nodes at an average rate $r$, and generates and writes data to nodes based on the read data. The distributed storage source capacity is the maximum amount of source that can be reliably stored for long periods of time. Previous research shows that asymptotically the distributed storage source capacity is at most $\left(1-\frac{e}{2 \cdot r}\right) \cdot N \cdot c$ as $N$ and $r$ grow. In this work we introduce and analyze algorithms such that asymptotically the distributed storage source data capacity is at least the above equation. Thus, the above equation expresses a fundamental trade-off between network traffic and storage overhead to reliably store source data.

📄 PDF Abstract BibTeX arXiv:2101.05223

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Distributed and Scalable Uplink Processing for LIS: Algorithm, Architecture, and Design Trade-offs

2020-12-09 · Jesus Rodriguez Sanchez, Fredrik Rusek, Ove Edfors, Liang Liu

The Large Intelligent Surface (LIS) is a promising technology in the areas of wireless communication, remote sensing and positioning. It consists of a continuous radiating surface located in the proximity of the users, w…

An Information Theoretic Approach to Distributed Representation Learning

2019-09-25 · Abdellatif Zaidi, Inaki Estella Aguerri

The problem of distributed representation learning is one in which multiple sources of information X1,...,XK are processed separately so as to extract useful information about some statistically correlated ground truth Y…

Representation LearningVariational Inference

Sample, computation vs storage tradeoffs for classification using tensor subspace models

2017-06-18 · Mohammadhossein Chaghazardi, Shuchin Aeron

In this paper, we exhibit the tradeoffs between the (training) sample, computation and storage complexity for the problem of supervised classification using signal subspace estimation. Our main tool is the use of tensor …

General Classification

Dynamic Privacy For Distributed Machine Learning Over Network

2016-01-14 · Tao Zhang, Quanyan Zhu

Privacy-preserving distributed machine learning becomes increasingly important due to the recent rapid growth of data. This paper focuses on a class of regularized empirical risk minimization (ERM) machine learning probl…

BIG-bench Machine LearningPrivacy Preserving

Near Optimal Coded Data Shuffling for Distributed Learning

2018-01-05 · Mohamed A. Attia, Ravi Tandon

Data shuffling between distributed cluster of nodes is one of the critical steps in implementing large-scale learning algorithms. Randomly shuffling the data-set among a cluster of workers allows different nodes to obtai…