Model-Attentive Ensemble Learning for Sequence Modeling
Medical time-series datasets have unique characteristics that make prediction tasks challenging. Most notably, patient trajectories often contain longitudinal variations in their input-output relationships, generally referred to as temporal conditional shift. Designing sequence models capable of adapting to such time-varying distributions remains a prevailing problem. To address this we present Model-Attentive Ensemble learning for Sequence modeling (MAES). MAES is a mixture of time-series experts which leverages an attention-based gating mechanism to specialize the experts on different sequence dynamics and adaptively weight their predictions. We demonstrate that MAES significantly out-performs popular sequence models on datasets subject to temporal shift.
Code (0)
등록된 구현이 없습니다.
Tasks
Ensemble LearningmodelTime SeriesTime Series AnalysisSimilar Papers 제목 키워드 기반
Exploring the sequence length bottleneck in the Transformer for Image Captioning
Most recent state of the art architectures rely on combinations and variations of three approaches: convolutional, recurrent and self-attentive methods. Our work attempts in laying the basis for a new research direction …
Image CaptioningSelf-Attentive Ensemble Transformer: Representing Ensemble Interactions in Neural Networks for Earth System Models
Ensemble data from Earth system models has to be calibrated and post-processed. I propose a novel member-by-member post-processing approach with neural networks. I bridge ideas from ensemble data assimilation with self-a…
Modeling Sequences as Star Graphs to Address Over-smoothing in Self-attentive Sequential Recommendation
Self-attention (SA) mechanisms have been widely used in developing sequential recommendation (SR) methods, and demonstrated state-of-the-art performance. However, in this paper, we show that self-attentive SR methods sub…
Sequential RecommendationMambaIRv2: Attentive State Space Restoration
The Mamba-based image restoration backbones have recently demonstrated significant potential in balancing global reception and computational efficiency. However, the inherent causal modeling limitation of Mamba, where ea…
Computational EfficiencyImage RestorationMambaSparse Attentive Memory Network for Click-through Rate Prediction with Long Sequences
Sequential recommendation predicts users' next behaviors with their historical interactions. Recommending with longer sequences improves recommendation accuracy and increases the degree of personalization. As sequences g…
Click-Through Rate PredictionRecommendation SystemsSequential Recommendation