Test Time Learning for Time Series Forecasting
Time-series forecasting has seen significant advancements with the introduction of token prediction mechanisms such as multi-head attention. However, these methods often struggle to achieve the same performance as in language modeling, primarily due to the quadratic computational cost and the complexity of capturing long-range dependencies in time-series data. State-space models (SSMs), such as Mamba, have shown promise in addressing these challenges by offering efficient solutions with linear RNNs capable of modeling long sequences with larger context windows. However, there remains room for improvement in accuracy and scalability. We propose the use of Test-Time Training (TTT) modules in a parallel architecture to enhance performance in long-term time series forecasting. Through extensive experiments on standard benchmark datasets, we demonstrate that TTT modules consistently outperform state-of-the-art models, including the Mamba-based TimeMachine, particularly in scenarios involving extended sequence and prediction lengths. Our results show significant improvements in Mean Squared Error (MSE) and Mean Absolute Error (MAE), especially on larger datasets such as Electricity, Traffic, and Weather, underscoring the effectiveness of TTT in capturing long-range dependencies. Additionally, we explore various convolutional architectures within the TTT framework, showing that even simple configurations like 1D convolution with small filters can achieve competitive results. This work sets a new benchmark for time-series forecasting and lays the groundwork for future research in scalable, high-performance forecasting models.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingMambaState Space ModelsTime SeriesTime Series ForecastingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
TimeGym: Debugging for Time Series Modeling in Python
We introduce the TimeGym Forecasting Debugging Toolkit, a Python library for testing and debugging time series forecasting pipelines. TimeGym simplifies the testing forecasting pipeline by providing generic tests for for…
test driven developmentTime SeriesTime Series AnalysisTime Series ForecastingFew-shot Learning for Time-series Forecasting
Time-series forecasting is important for many applications. Forecasting models are usually trained using time-series data in a specific target task. However, sufficient data in the target task might be unavailable, which…
Few-Shot LearningTime SeriesTime Series AnalysisTime Series ForecastingSelf-Adaptive Forecasting for Improved Deep Learning on Non-Stationary Time-Series
Real-world time-series datasets often violate the assumptions of standard supervised learning for forecasting -- their distributions evolve over time, rendering the conventional training and model selection procedures su…
DecoderModel SelectionSelf-Supervised LearningTime Series+2Empowering Time Series Forecasting with LLM-Agents
Large Language Model (LLM) powered agents have emerged as effective planners for Automated Machine Learning (AutoML) systems. While most existing AutoML approaches focus on automating feature engineering and model archit…
Time Series ForecastingFeature EngineeringCorrelated daily time series and forecasting in the M4 competition
We participated in the M4 competition for time series forecasting and describe here our methods for forecasting daily time series. We used an ensemble of five statistical forecasting methods and a method that we refer to…
Time SeriesTime Series AnalysisTime Series Forecasting