Small Language Models as Compiler Experts: Auto-Parallelization for Heterogeneous Systems
Traditional auto-parallelizing compilers, reliant on rigid heuristics, struggle with the complexity of modern heterogeneous systems. This paper presents a comprehensive evaluation of small (approximately 1B parameter) language-model-driven compiler auto-parallelization. We evaluate three models: gemma3, llama3.2, and qwen2.5, using six reasoning strategies across 11 real-world kernels drawn from scientific computing, graph algorithms, and machine learning. Our system is benchmarked against strong compiler baselines, including LLVM Polly, TVM, and Triton. Across 376 total evaluations, the proposed approach achieves an average speedup of 6.81x and a peak performance of 43.25x on convolution operations. We analyze scalability, verify correctness using multiple sanitizers, and confirm robustness across diverse compilers and hardware platforms. Our results demonstrate that small, efficient language models can serve as powerful reasoning engines for complex compiler optimization tasks.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Learning to Parallelize in a Shared-Memory Environment with Transformers
In past years, the world has switched to many-core and multi-core shared memory architectures. As a result, there is a growing need to utilize these architectures by introducing shared memory parallelization schemes to s…
ManagementAdvising OpenMP Parallelization via a Graph-Based Approach with Transformers
There is an ever-present need for shared memory parallelization schemes to exploit the full potential of multi-core architectures. The most common parallelization API addressing this need today is OpenMP. Nevertheless, w…
Data AugmentationOMPar: Automatic Parallelization with AI-Driven Source-to-Source Compilation
Manual parallelization of code remains a significant challenge due to the complexities of modern software systems and the widespread adoption of multi-core architectures. This paper introduces OMPar, an AI-driven tool de…
C++ codeTowards Neural Decompilation
We address the problem of automatic decompilation, converting a program in low-level representation back to a higher-level human-readable programming language. The problem of decompilation is extremely important for secu…
C++ codeMachine TranslationTranslationGenerating GPU Compiler Heuristics using Reinforcement Learning
GPU compilers are complex software programs with many optimizations specific to target hardware. These optimizations are often controlled by heuristics hand-designed by compiler experts using time- and resource-intensive…
Deep Reinforcement LearningGPUreinforcement-learningReinforcement Learning+1