Residual Attention Net for Superior Cross-Domain Time Sequence Modeling
We present a novel architecture, residual attention net (RAN), which merges a sequence architecture, universal transformer, and a computer vision architecture, residual net, with a high-way architecture for cross-domain sequence modeling. The architecture aims at addressing the long dependency issue often faced by recurrent-neural-net-based structures. This paper serves as a proof-of-concept for a new architecture, with RAN aiming at providing the model a higher level understanding of sequence patterns. To our best knowledge, we are the first to propose such an architecture. Out of the standard 85 UCR data sets, we have achieved 35 state-of-the-art results with 10 results matching current state-of-the-art results without further model fine-tuning. The results indicate that such architecture is promising in complex, long-sequence modeling and may have vast, cross-domain applications.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Attention Residual Fusion Network with Contrast for Source-free Domain Adaptation
Source-free domain adaptation (SFDA) involves training a model on source domain and then applying it to a related target domain without access to the source data and labels during adaptation. The complexity of scene info…
Source-Free Domain AdaptationADOD: Adaptive Domain-Aware Object Detection with Residual Attention for Underwater Environments
This research presents ADOD, a novel approach to address domain generalization in underwater object detection. Our method enhances the model's ability to generalize across diverse and unseen domains, ensuring robustness …
domain classificationDomain GeneralizationObjectobject-detection+1Cross-attention Secretly Performs Orthogonal Alignment in Recommendation Models
Cross-domain sequential recommendation (CDSR) aims to align heterogeneous user behavior sequences collected from different domains. While cross-attention is widely used to enhance alignment and improve recommendation per…
Sequential Recommendation3D-DDA: 3D Dual-Domain Attention for Brain Tumor Segmentation
Accurate brain tumor segmentation plays an essential role in the diagnosis process. However, there are challenges due to the variety of tumors in low contrast, morphology, location, annotation bias, and imbalance among t…
Brain Tumor SegmentationTumor SegmentationQuantum Adaptive Self-Attention for Quantum Transformer Models
Transformer models have revolutionized sequential learning across various domains, yet their self-attention mechanism incurs quadratic computational cost, posing limitations for real-time and resource-constrained tasks. …