Positional Encoding in Transformer-Based Time Series Models: A Survey
Recent advancements in transformer-based models have greatly improved time series analysis, providing robust solutions for tasks such as forecasting, anomaly detection, and classification. A crucial element of these models is positional encoding, which allows transformers to capture the intrinsic sequential nature of time series data. This survey systematically examines existing techniques for positional encoding in transformer-based time series models. We investigate a variety of methods, including fixed, learnable, relative, and hybrid approaches, and evaluate their effectiveness in different time series classification tasks. Furthermore, we outline key challenges and suggest potential research directions to enhance positional encoding strategies. By delivering a comprehensive overview and quantitative benchmarking, this survey intends to assist researchers and practitioners in selecting and designing effective positional encoding methods for transformer-based time series models.
Code (1)
Tasks
Anomaly DetectionBenchmarkingSurveyTime SeriesTime Series AnalysisTime Series ClassificationSimilar Papers 제목 키워드 기반
DyWPE: Signal-Aware Dynamic Wavelet Positional Encoding for Time Series Transformers
Existing positional encoding methods in transformers are fundamentally signal-agnostic, deriving positional information solely from sequence indices while ignoring the underlying signal characteristics. This limitation i…
Time Series AnalysisImproving Transformers using Faithful Positional Encoding
We propose a new positional encoding method for a neural network architecture called the Transformer. Unlike the standard sinusoidal positional encoding, our approach is based on solid mathematical grounds and has a guar…
Time SeriesTime Series ClassificationContinuous-Time Linear Positional Embedding for Irregular Time Series Forecasting
Irregularly sampled time series forecasting, characterized by non-uniform intervals, is prevalent in practical applications. However, previous research have been focused on regular time series forecasting, typically rely…
Irregular Time SeriesTime SeriesTime Series ForecastingExploring the Role of Token in Transformer-based Time Series Forecasting
Transformer-based methods are a mainstream approach for solving time series forecasting (TSF). These methods use temporal or variable tokens from observable data to make predictions. However, most focus on optimizing the…
Time SeriesTime Series ForecastingTransformers in Time-series Analysis: A Tutorial
Transformer architecture has widespread applications, particularly in Natural Language Processing and computer vision. Recently Transformers have been employed in various aspects of time-series analysis. This tutorial pr…
DecoderTime SeriesTime Series Analysis