TSLANet: Rethinking Transformers for Time Series Representation Learning
Time series data, characterized by its intrinsic long and short-range dependencies, poses a unique challenge across analytical applications. While Transformer-based models excel at capturing long-range dependencies, they face limitations in noise sensitivity, computational efficiency, and overfitting with smaller datasets. In response, we introduce a novel Time Series Lightweight Adaptive Network (TSLANet), as a universal convolutional model for diverse time series tasks. Specifically, we propose an Adaptive Spectral Block, harnessing Fourier analysis to enhance feature representation and to capture both long-term and short-term interactions while mitigating noise via adaptive thresholding. Additionally, we introduce an Interactive Convolution Block and leverage self-supervised learning to refine the capacity of TSLANet for decoding complex temporal patterns and improve its robustness on different datasets. Our comprehensive experiments demonstrate that TSLANet outperforms state-of-the-art models in various tasks spanning classification, forecasting, and anomaly detection, showcasing its resilience and adaptability across a spectrum of noise levels and data sizes. The code is available at https://github.com/emadeldeen24/TSLANet.
Code (2)
Tasks
Anomaly DetectionComputational EfficiencyRepresentation LearningSelf-Supervised LearningTime SeriesTime Series ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
UnitNorm: Rethinking Normalization for Transformers in Time Series
Normalization techniques are crucial for enhancing Transformer models' performance and stability in time series analysis tasks, yet traditional methods like batch and layer normalization often lead to issues such as toke…
Anomaly DetectionTime SeriesTime Series AnalysisFusing Large Language Models with Temporal Transformers for Time Series Forecasting
Recently, large language models (LLMs) have demonstrated powerful capabilities in performing various tasks and thus are applied by recent studies to time series forecasting (TSF) tasks, which predict future values with t…
Time Series ForecastingTime Series PredictionAttention Augmented Convolutional Transformer for Tabular Time-series
Time-series classification is one of the most frequently performed tasks in industrial data science, and one of the most widely used data representation in the industrial setting is tabular representation. In this work, …
Language ModelingLanguage ModellingQuantizationRepresentation Learning+3Time Series Representations for Classification Lie Hidden in Pretrained Vision Transformers
Time series classification is a fundamental task in healthcare and industry, yet the development of time series foundation models (TSFMs) remains limited by the scarcity of publicly available time series datasets. In thi…
Time SeriesTime Series ClassificationW-Transformers : A Wavelet-based Transformer Framework for Univariate Time Series Forecasting
Deep learning utilizing transformers has recently achieved a lot of success in many vital areas such as natural language processing, computer vision, anomaly detection, and recommendation systems, among many others. Amon…
Anomaly DetectionRecommendation SystemsRepresentation LearningTime Series+3