Minusformer: Improving Time Series Forecasting by Progressively Learning Residuals
In this paper, we find that ubiquitous time series (TS) forecasting models are prone to severe overfitting. To cope with this problem, we embrace a de-redundancy approach to progressively reinstate the intrinsic values of TS for future intervals. Specifically, we introduce a dual-stream and subtraction mechanism, which is a deep Boosting ensemble learning method. And the vanilla Transformer is renovated by reorienting the information aggregation mechanism from addition to subtraction. Then, we incorporate an auxiliary output branch into each block of the original model to construct a highway leading to the ultimate prediction. The output of subsequent modules in this branch will subtract the previously learned results, enabling the model to learn the residuals of the supervision signal, layer by layer. This designing facilitates the learning-driven implicit progressive decomposition of the input and output streams, empowering the model with heightened versatility, interpretability, and resilience against overfitting. Since all aggregations in the model are minus signs, which is called Minusformer. Extensive experiments demonstrate the proposed method outperform existing state-of-the-art methods, yielding an average performance improvement of 11.9% across various datasets.The code has been released at https://github.com/Anoise/Minusformer.
Code (1)
Tasks
Ensemble LearningTime SeriesTime Series AnalysisTime Series ForecastingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
DeepBooTS: Dual-Stream Residual Boosting for Drift-Resilient Time-Series Forecasting
Time-Series (TS) exhibits pronounced non-stationarity. Consequently, most forecasting methods display compromised robustness to concept drift, despite the prevalent application of instance normalization. We tackle this c…
Signals, Concepts, and Laws: Toward Universal, Explainable Time-Series Forecasting
Accurate, explainable and physically credible forecasting remains a persistent challenge for multivariate time-series whose statistical properties vary across domains. We propose DORIC, a Domain-Universal, ODE-Regularize…
TEFL: Prediction-Residual-Guided Rolling Forecasting for Multi-Horizon Time Series
Time series forecasting plays a critical role in domains such as transportation, energy, and meteorology. Despite their success, modern deep forecasting models are typically trained to minimize point-wise prediction loss…
Time Series ForecastingBoosted Ensemble Learning based on Randomized NNs for Time Series Forecasting
Time series forecasting is a challenging problem particularly when a time series expresses multiple seasonality, nonlinear trend and varying variance. In this work, to forecast complex time series, we propose ensemble le…
Ensemble LearningTime SeriesTime Series AnalysisTime Series ForecastingExplainable boosted linear regression for time series forecasting
Time series forecasting involves collecting and analyzing past observations to develop a model to extrapolate such observations into the future. Forecasting of future events is important in many fields to support decisio…
Decision MakingPrediction IntervalsregressionTime Series+2