Custom Algorithm-based Fault Tolerance for Attention Layers in Transformers

2025-07-22 · Vasileios Titopoulos, Kosmas Alexandridis, Giorgos Dimitrakopoulos arxiv

Transformers and large language models (LLMs), powered by the attention mechanism, have transformed numerous AI applications, driving the need for specialized hardware accelerators. A major challenge in these accelerators is efficiently detecting errors caused by random hardware faults. Traditional algorithm-based fault tolerance (ABFT) techniques verify individual matrix multiplications but fall short in handling the full attention mechanism, particularly due to intermediate softmax normalization. This work proposes Flash-ABFT, a novel method that computes an online checksum across the entire three-matrix product of query, key and value matrices, of an attention layer, including the softmax operation, with a single check. This approach significantly reduces overhead by eliminating redundant checks while maintaining high fault-detection accuracy. Experimental results demonstrate that Flash-ABFT incurs only 5.3% hardware area overhead and less than 1.9% energy overhead, making it a cost-effective and robust solution for error detection in attention accelerators.

📄 PDF Abstract BibTeX arXiv:2507.16676

Code (0)

등록된 구현이 없습니다.

Arithmetic-Intensity-Guided Fault Tolerance for Neural Network Inference on GPUs

2021-04-19 · Jack Kosaian, K. V. Rashmi

Neural networks (NNs) are increasingly employed in safety-critical domains and in environments prone to unreliability (e.g., soft errors), such as on spacecraft. Therefore, it is critical to impart fault tolerance to NN …

FT-Transformer: Resilient and Reliable Transformer with End-to-End Fault Tolerant Attention

2025-04-03 · Huangliang Dai, Shixun Wu, Hairui Zhao, Jiajun Huang 외

Transformer models leverage self-attention mechanisms to capture complex dependencies, demonstrating exceptional performance in various applications. However, the long-duration high-load computations required for model i…

Exploring Winograd Convolution for Cost-effective Neural Network Fault Tolerance

2023-08-16 · Xinghua Xue, Cheng Liu, Bo Liu, Haitong Huang 외

Winograd is generally utilized to optimize convolution performance and computational efficiency because of the reduced multiplication operations, but the reliability issues brought by winograd are usually overlooked. In …

Computational Efficiency

Byzantine Fault-Tolerance in Federated Local SGD under 2f-Redundancy

2021-08-26 · Nirupam Gupta, Thinh T. Doan, Nitin Vaidya

We consider the problem of Byzantine fault-tolerance in federated machine learning. In this problem, the system comprises multiple agents each with local data, and a trusted centralized coordinator. In fault-free setting…

Towards Fault Tolerance in Multi-Agent Reinforcement Learning

2024-11-30 · Yuchen Shi, Huaxin Pei, Liang Feng, Yi Zhang 외

Agent faults pose a significant threat to the performance of multi-agent reinforcement learning (MARL) algorithms, introducing two key challenges. First, agents often struggle to extract critical information from the cha…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning