Enhancing Time Series Forecasting with Fuzzy Attention-Integrated Transformers

2025-03-31 · Sanjay Chakraborty, Fredrik Heintz

This paper introduces FANTF (Fuzzy Attention Network-Based Transformers), a novel approach that integrates fuzzy logic with existing transformer architectures to advance time series forecasting, classification, and anomaly detection tasks. FANTF leverages a proposed fuzzy attention mechanism incorporating fuzzy membership functions to handle uncertainty and imprecision in noisy and ambiguous time series data. The FANTF approach enhances its ability to capture complex temporal dependencies and multivariate relationships by embedding fuzzy logic principles into the self-attention module of the existing transformer's architecture. The framework combines fuzzy-enhanced attention with a set of benchmark existing transformer-based architectures to provide efficient predictions, classification and anomaly detection. Specifically, FANTF generates learnable fuzziness attention scores that highlight the relative importance of temporal features and data points, offering insights into its decision-making process. Experimental evaluatios on some real-world datasets reveal that FANTF significantly enhances the performance of forecasting, classification, and anomaly detection tasks over traditional transformer-based models.

📄 PDF Abstract BibTeX arXiv:2504.00070

Code (1)

sanjaylopa22/FANTF 공식 구현 pytorch

Tasks

Anomaly Detection Decision MakingTime SeriesTime Series Forecasting

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Attention 설명 없음

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

A Neuro-Fuzzy System for Interpretable Long-Term Stock Market Forecasting

2025-10-01 · Miha Ožbot, Igor Škrjanc, Vitomir Štruc arxiv

In the complex landscape of multivariate time series forecasting, achieving both accuracy and interpretability remains a significant challenge. This paper introduces the Fuzzy Transformer (Fuzzformer), a novel recurrent …

Multivariate Time Series Forecasting

Causal Graph Fuzzy LLMs: A First Introduction and Applications in Time Series Forecasting

2025-07-22 · Omid Orang, Patricia O. Lucas, Gabriel I. F. Paiva, Petronio C. L. Silva 외 arxiv

In recent years, the application of Large Language Models (LLMs) to time series forecasting (TSF) has garnered significant attention among researchers. This study presents a new frame of LLMs named CGF-LLM using GPT-2 co…

Time Series Forecasting

Differential Convolutional Fuzzy Time Series Forecasting

2023-05-15 · Tianxiang Zhan, Yuanpeng He, Yong Deng, Zhen Li

Fuzzy time series forecasting (FTSF) is a typical forecasting method with wide application. Traditional FTSF is regarded as an expert system which leads to loss of the ability to recognize undefined features. The mention…

Time Series Time Series Forecasting

A novel method of fuzzy time series forecasting based on interval index number and membership value using support vector machine

2020-10-20 · Kiran Bisht, Arun Kumar

Fuzzy time series forecasting methods are very popular among researchers for predicting future values as they are not based on the strict assumptions of traditional time series forecasting methods. Non-stochastic methods…

Clustering Time Series Time Series Analysis Time Series Forecasting

FISformer: Replacing Self-Attention with a Fuzzy Inference System in Transformer Models for Time Series Forecasting

2026-03-23 · Bulent Haznedar, Levent Karacan arxiv

Transformers have achieved remarkable progress in time series forecasting, yet their reliance on deterministic dot-product attention limits their capacity to model uncertainty and nonlinear dependencies across multivaria…