홈 › Papers

swTVM: Towards Optimized Tensor Code Generation for Deep Learning on Sunway Many-Core Processor

2019-04-16 · Mingzhen Li, Changxi Liu, Jianjin Liao, Xuegui Zheng, Hailong Yang, Rujun Sun, Jun Xu, Lin Gan, Guangwen Yang, Zhongzhi Luan, Depei Qian

The flourish of deep learning frameworks and hardware platforms has been demanding an efficient compiler that can shield the diversity in both software and hardware in order to provide application portability. Among the existing deep learning compilers, TVM is well known for its efficiency in code generation and optimization across diverse hardware devices. In the meanwhile, the Sunway many-core processor renders itself as a competitive candidate for its attractive computational power in both scientific computing and deep learning workloads. This paper combines the trends in these two directions. Specifically, we propose swTVM that extends the original TVM to support ahead-of-time compilation for architecture requiring cross-compilation such as Sunway. In addition, we leverage the architecture features during the compilation such as core group for massive parallelism, DMA for high bandwidth memory transfer and local device memory for data locality, in order to generate efficient codes for deep learning workloads on Sunway. The experiment results show that the codes generated by swTVM achieves 1.79x on average compared to the state-of-the-art deep learning framework on Sunway, across six representative benchmarks. This work is the first attempt from the compiler perspective to bridge the gap of deep learning and Sunway processor particularly with productivity and efficiency in mind. We believe this work will encourage more people to embrace the power of deep learning and Sunway many-core processor.

📄 PDF Abstract BibTeX arXiv:1904.07404

Code (0)

등록된 구현이 없습니다.

Tasks

Code GenerationDeep Learning

Methods 이 논문이 사용한 방법론

1x1 Convolution A 1 x 1 Convolution is a convolution with some special properties in that it can be used for dimensionality reduction,…

Local Response Normalization Local Response Normalization is a normalization layer that implements the idea of lateral inhibition. Lateral inhibition is a concept in neurobiology that refers to the…

Grouped Convolution A Grouped Convolution uses a group of convolutions - multiple kernels per layer - resulting in multiple channel outputs per layer. This leads to wider networks helping a…

ReLU How Do I Communicate to Expedia? How Do I Communicate to Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Live Support & Special Travel…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Max Pooling Max Pooling is a pooling operation that calculates the maximum value for patches of a feature map, and uses it to create a downsampled (pooled) feature map. It is usually…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

CodegenBench: Can LLMs Write Efficient Code Across Architectures?

2026-06-01 · Jie Li, Wenzhao Wu, Junqi Hu, Qinrui Zheng 외 arxiv

While large language models (LLMs) have been extensively evaluated on code generation tasks for general-purpose programming and GPU-accelerated environments (e.g., PyTorch, CUDA), their capabilities in CPU-oriented high-…

Code Generation

CoMERA: Computing- and Memory-Efficient Training via Rank-Adaptive Tensor Optimization

2024-05-23 · Zi Yang, Ziyue Liu, Samridhi Choudhary, Xinfeng Xie 외

Training large AI models such as LLMs and DLRMs costs massive GPUs and computing time. The high training cost has become only affordable to big tech companies, meanwhile also causing increasing concerns about the environ…

Code Generation GPU Recommendation Systems

The CoRa Tensor Compiler: Compilation for Ragged Tensors with Minimal Padding

2021-10-19 · Pratik Fegade, Tianqi Chen, Phillip B. Gibbons, Todd C. Mowry

There is often variation in the shape and size of input data used for deep learning. In many cases, such data can be represented using tensors with non-uniform shapes, or ragged tensors. Due to limited and non-portable s…

CPU GPU tensor algebra

swCaffe: a Parallel Framework for Accelerating Deep Learning Applications on Sunway TaihuLight

2019-03-16 · Jiarui Fang, Liandeng Li, Haohuan Fu, Jinlei Jiang 외

This paper reports our efforts on swCaffe, a highly efficient parallel framework for accelerating deep neural networks (DNNs) training on Sunway TaihuLight, the current fastest supercomputer in the world that adopts a un…

CPU GPU

Deep Optimized Multiple Description Image Coding via Scalar Quantization Learning

2020-01-12 · Lijun Zhao, Huihui Bai, Anhong Wang, Yao Zhao

In this paper, we introduce a deep multiple description coding (MDC) framework optimized by minimizing multiple description (MD) compressive loss. First, MD multi-scale-dilated encoder network generates multiple descript…

Decoder Quantization