paper-with-me

홈 › Papers

TFBEST: Dual-Aspect Transformer with Learnable Positional Encoding for Failure Prediction

2023-09-06 · Rohan Mohapatra, Saptarshi Sengupta

Hard Disk Drive (HDD) failures in datacenters are costly - from catastrophic data loss to a question of goodwill, stakeholders want to avoid it like the plague. An important tool in proactively monitoring against HDD failure is timely estimation of the Remaining Useful Life (RUL). To this end, the Self-Monitoring, Analysis and Reporting Technology employed within HDDs (S.M.A.R.T.) provide critical logs for long-term maintenance of the security and dependability of these essential data storage devices. Data-driven predictive models in the past have used these S.M.A.R.T. logs and CNN/RNN based architectures heavily. However, they have suffered significantly in providing a confidence interval around the predicted RUL values as well as in processing very long sequences of logs. In addition, some of these approaches, such as those based on LSTMs, are inherently slow to train and have tedious feature engineering overheads. To overcome these challenges, in this work we propose a novel transformer architecture - a Temporal-fusion Bi-encoder Self-attention Transformer (TFBEST) for predicting failures in hard-drives. It is an encoder-decoder based deep learning technique that enhances the context gained from understanding health statistics sequences and predicts a sequence of the number of days remaining before a disk potentially fails. In this paper, we also provide a novel confidence margin statistic that can help manufacturers replace a hard-drive within a time frame. Experiments on Seagate HDD data show that our method significantly outperforms the state-of-the-art RUL prediction methods during testing over the exhaustive 10-year data from Backblaze (2013-present). Although validated on HDD failure prediction, the TFBEST architecture is well-suited for other prognostics applications and may be adapted for allied regression problems.

📄 PDF Abstract BibTeX arXiv:2309.02641

Code (0)

등록된 구현이 없습니다.

Tasks

Feature Engineering

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음
Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Residual Connection 설명 없음
Adam 설명 없음
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Similar Papers 제목 키워드 기반

Learnable Spatial-Temporal Positional Encoding for Link Prediction

2025-06-10 · Katherine Tieu, Dongqi Fu, Zihao Li, Ross Maciejewski 외

Accurate predictions rely on the expressiveness power of graph deep learning frameworks like graph neural networks and graph transformers, where a positional encoding mechanism has become much more indispensable in recen…

Link PredictionPrediction

LoPE: Learnable Sinusoidal Positional Encoding for Improving Document Transformer Model

2022-01-16 · ACL ARR January 2022 1 · Anonymous

Positional encoding plays a key role in Transformer-based architecture, which is to indicate and embed token sequential order information. Understanding documents with unreliable reading order information is a real chall…

document understanding

Learning to Iteratively Solve Routing Problems with Dual-Aspect Collaborative Transformer

2021-10-06 · NeurIPS 2021 12 · Yining Ma, Jingwen Li, Zhiguang Cao, Wen Song 외

Recently, Transformer has become a prevailing deep architecture for solving vehicle routing problems (VRPs). However, it is less effective in learning improvement models for VRP because its positional encoding (PE) metho…

Traveling Salesman Problem

Graph Neural Networks with Learnable Structural and Positional Representations

2021-10-15 · ICLR 2022 4 · Vijay Prakash Dwivedi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio 외

Graph neural networks (GNNs) have become the standard learning architectures for graphs. GNNs have been applied to numerous domains ranging from quantum chemistry, recommender systems to knowledge graphs and natural lang…

Graph RegressionKnowledge GraphsRecommendation Systems

Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding

2021-06-05 · NeurIPS 2021 12 · Yang Li, Si Si, Gang Li, Cho-Jui Hsieh 외

Attentional mechanisms are order-invariant. Positional encoding is a crucial component to allow attention-based deep model architectures such as Transformer to address sequences or images where the position of informatio…

Position