Optimally Scheduling CNN Convolutions for Efficient Memory Access

2019-02-04 · Arthur Stoutchinin, Francesco Conti, Luca Benini

Embedded inference engines for convolutional networks must be parsimonious in memory bandwidth and buffer sizing to meet power and cost constraints. We present an analytical memory bandwidth model for loop-nest optimization targeting architectures with application managed buffers. We applied this model to optimize the CNN convolution loop-nest. We show that our model is more accurate than previously published models. Using this model we can identify non-trivial dataflow schedules that result in lowest communication bandwidth given tight local buffering constraints. We show that optimal dataflow schedules are implementable in practice and that our model is accurate with respect to a real implementation; moreover, we introduce an accelerator architecture, named Hardware Convolution Block (HWC), which implements the optimal schedules, and we show it achieves up to 14x memory bandwidth reduction compared to a previously published accelerator with a similar memory interface, but implementing a non-optimal schedule.

📄 PDF Abstract BibTeX arXiv:1902.01492

Code (0)

등록된 구현이 없습니다.

Tasks

Scheduling

Methods 이 논문이 사용한 방법론

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Optimizing Grouped Convolutions on Edge Devices

2020-06-17 · Perry Gibson, José Cano, Jack Turner, Elliot J. Crowley 외

When deploying a deep neural network on constrained hardware, it is possible to replace the network's standard convolutions with grouped convolutions. This allows for substantial memory savings with minimal loss of accur…

CADS: Core-Aware Dynamic Scheduler for Multicore Memory Controllers

2019-07-17 · Eduardo Olmedo Sanchez, Xian-He Sun

Memory controller scheduling is crucial in multicore processors, where DRAM bandwidth is shared. Since increased number of requests from multiple cores of processors becomes a source of bottleneck, scheduling the request…

Fairness Reinforcement Learning Reinforcement Learning (RL)Scheduling

Optimizing Attention on GPUs by Exploiting GPU Architectural NUMA Effects

2025-11-03 · Mansi Choudhary, Karthik Sangaiah, Sonali Singh, Muhammad Osama 외 arxiv

The rise of disaggregated AI GPUs has exposed a critical bottleneck in large-scale attention workloads: non-uniform memory access (NUMA). As multi-chiplet designs become the norm for scaling compute capabilities, memory …

Optimizing LLM Inference: Fluid-Guided Online Scheduling with Memory Constraints

2025-04-15 · Ruicheng Ao, Gan Luo, David Simchi-Levi, Xinshang Wang

Large Language Models (LLMs) are indispensable in today's applications, but their inference procedure -- generating responses by processing text in segments and using a memory-heavy Key-Value (KV) cache -- demands signif…

GPU Inference Optimization Scheduling

HUGE2: a Highly Untangled Generative-model Engine for Edge-computing

2019-07-25 · Feng Shi, Ziheng Xu, Tao Yuan, Song-Chun Zhu

As a type of prominent studies in deep learning, generative models have been widely investigated in research recently. Two research branches of the deep learning models, the Generative Networks (GANs, VAE) and the Semant…

Deep Learning Edge-computing Semantic Segmentation