Memory Safe Computations with XLA Compiler

2022-06-28 · Artem Artemev, Tilman Roeder, Mark van der Wilk

Software packages like TensorFlow and PyTorch are designed to support linear algebra operations, and their speed and usability determine their success. However, by prioritising speed, they often neglect memory requirements. As a consequence, the implementations of memory-intensive algorithms that are convenient in terms of software design can often not be run for large problems due to memory overflows. Memory-efficient solutions require complex programming approaches with significant logic outside the computational framework. This impairs the adoption and use of such algorithms. To address this, we developed an XLA compiler extension that adjusts the computational data-flow representation of an algorithm according to a user-specified memory limit. We show that k-nearest neighbour and sparse Gaussian process regression methods can be run at a much larger scale on a single device, where standard implementations would have failed. Our approach leads to better use of hardware resources. We believe that further focus on removing memory constraints at a compiler level will widen the range of machine learning methods that can be developed in the future.

📄 PDF Abstract BibTeX arXiv:2206.14148

Code (1)

awav/tensorflow 공식 구현 tf

Methods 이 논문이 사용한 방법론

SPEED The monocular depth estimation (MDE) is the task of estimating depth from a single frame. This information is an essential knowledge in many computer vision tasks such as scene…

Gaussian Process Gaussian Processes are non-parametric models for approximating functions. They rely upon a measure of similarity between points (the kernel function) to predict the value for…

FusionStitching: Boosting Memory Intensive Computations for Deep Learning Workloads

2020-09-23 · Zhen Zheng, Pengzhan Zhao, Guoping Long, Feiwen Zhu 외

We show in this work that memory intensive computations can result in severe performance problems due to off-chip memory access and CPU-GPU context switch overheads in a wide range of deep learning models. For this probl…

Code Generation CPU Deep Learning GPU

NGEMM: Optimizing GEMM for Deep Learning via Compiler-based Techniques

2019-10-01 · Wenlei Bao, Li-Wen Chang, Yang Chen, Ke Deng 외

Quantization has emerged to be an effective way to significantly boost the performance of deep neural networks (DNNs) by utilizing low-bit computations. Despite having lower numerical precision, quantized DNNs are able t…

Deep Learning Quantization

CHET: Compiler and Runtime for Homomorphic Evaluation of Tensor Programs

2018-10-01 · Roshan Dathathri, Olli Saarikivi, Hao Chen, Kim Laine 외

Fully Homomorphic Encryption (FHE) refers to a set of encryption schemes that allow computations to be applied directly on encrypted data without requiring a secret key. This enables novel application scenarios where a c…

CLSA-CIM: A Cross-Layer Scheduling Approach for Computing-in-Memory Architectures

2024-01-15 · Rebecca Pelke, Jose Cubero-Cascante, Nils Bosbach, Felix Staudigl 외

The demand for efficient machine learning (ML) accelerators is growing rapidly, driving the development of novel computing concepts such as resistive random access memory (RRAM)-based tiled computing-in-memory (CIM) arch…

Code Generation Scheduling

AMULET: Adaptive Matrix-Multiplication-Like Tasks

2023-05-12 · Junyoung Kim, Kenneth Ross, Eric Sedlar, Lukas Stadler

Many useful tasks in data science and machine learning applications can be written as simple variations of matrix multiplication. However, users have difficulty performing such tasks as existing matrix/vector libraries s…

Compiler Optimization