paper-with-me

홈 › Papers

Trends in AI Supercomputers

2025-04-22 · Konstantin F. Pilz, James Sanders, Robi Rahman, Lennart Heim

Frontier AI development relies on powerful AI supercomputers, yet analysis of these systems is limited. We create a dataset of 500 AI supercomputers from 2019 to 2025 and analyze key trends in performance, power needs, hardware cost, ownership, and global distribution. We find that the computational performance of AI supercomputers has doubled every nine months, while hardware acquisition cost and power needs both doubled every year. The leading system in March 2025, xAI's Colossus, used 200,000 AI chips, had a hardware cost of \$7B, and required 300 MW of power, as much as 250,000 households. As AI supercomputers evolved from tools for science to industrial machines, companies rapidly expanded their share of total AI supercomputer performance, while the share of governments and academia diminished. Globally, the United States accounts for about 75% of total performance in our dataset, with China in second place at 15%. If the observed trends continue, the leading AI supercomputer in 2030 will achieve $2\times10^{22}$ 16-bit FLOP/s, use two million AI chips, have a hardware cost of \$200 billion, and require 9 GW of power. Our analysis provides visibility into the AI supercomputer landscape, allowing policymakers to assess key AI trends like resource needs, ownership, and national competitiveness.

📄 PDF Abstract BibTeX arXiv:2504.16026

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Redesigning pattern mining algorithms for supercomputers

2015-10-27 · Kazuki Yoshizoe, Aika Terada, Koji Tsuda

Upcoming many core processors are expected to employ a distributed memory architecture similar to currently available supercomputers, but parallel pattern mining algorithms amenable to the architecture are not comprehens…

Inverse renormalization group of spin glasses

2023-10-19 · Dimitrios Bachtis

We propose inverse renormalization group transformations to construct approximate configurations for lattice volumes that have not yet been accessed by supercomputers or large-scale simulations in the study of spin glass…

Towards Automatic Learning of Heuristics for Mechanical Transformations of Procedural Code

2017-01-25 · Guillermo Vigueras, Manuel Carro, Salvador Tamarit, Julio Mariño

The current trends in next-generation exascale systems go towards integrating a wide range of specialized (co-)processors into traditional supercomputers. Due to the efficiency of heterogeneous systems in terms of Watts …

Reinforcement Learning

Exploring GPU-to-GPU Communication: Insights into Supercomputer Interconnects

2024-08-26 · Daniele De Sensi, Lorenzo Pichetti, Flavio Vella, Tiziano De Matteis 외

Multi-GPU nodes are increasingly common in the rapidly evolving landscape of exascale supercomputers. On these systems, GPUs on the same node are connected through dedicated networks, with bandwidths up to a few terabits…

GPU

The Big Send-off: High Performance Collectives on GPU-based Supercomputers

2025-04-25 · Siddharth Singh, Mahua Singh, Abhinav Bhatele

We evaluate the current state of collective communication on GPU-based supercomputers for large language model (LLM) training at scale. Existing libraries such as RCCL and Cray-MPICH exhibit critical limitations on syste…

GPULanguage ModelingLanguage ModellingLarge Language Model