paper-with-me

Papers

IncepFormerNet: A multi-scale multi-head attention network for SSVEP classification

2025-02-04 · Yan Huang, Yongru Chen, Lei Cao, Yongnian Cao, Xuechun yang, Yilin Dong, Tianyu Liu

In recent years, deep learning (DL) models have shown outstanding performance in EEG classification tasks, particularly in Steady-State Visually Evoked Potential(SSVEP)-based Brain-Computer-Interfaces(BCI)systems. DL methods have been successfully applied to SSVEP-BCI. This study proposes a new model called IncepFormerNet, which is a hybrid of the Inception and Transformer architectures. IncepFormerNet adeptly extracts multi-scale temporal information from time series data using parallel convolution kernels of varying sizes, accurately capturing the subtle variations and critical features within SSVEP signals.Furthermore, the model integrates the multi-head attention mechanism from the Transformer architecture, which not only provides insights into global dependencies but also significantly enhances the understanding and representation of complex patterns.Additionally, it takes advantage of filter bank techniques to extract features based on the spectral characteristics of SSVEP data. To validate the effectiveness of the proposed model, we conducted experiments on two public datasets, . The experimental results show that IncepFormerNet achieves an accuracy of 87.41 on Dataset 1 and 71.97 on Dataset 2 using a 1.0-second time window. To further verify the superiority of the proposed model, we compared it with other deep learning models, and the results indicate that our method achieves significantly higher accuracy than the others.The source codes in this work are available at: https://github.com/CECNL/SSVEP-DAN.

📄 PDF Abstract BibTeX arXiv:2502.13972

Code (1)

cecnl/ssvep-dan 공식 구현 pytorch

Tasks

EEGSSVEP

Methods 이 논문이 사용한 방법론

Attention 설명 없음
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
Residual Connection 설명 없음
Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…
Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Similar Papers 제목 키워드 기반

Multiscaled Multi-Head Attention-based Video Transformer Network for Hand Gesture Recognition

2025-01-01 · Mallika Garg, Debashis Ghosh, Pyari Mohan Pradhan

Dynamic gesture recognition is one of the challenging research areas due to variations in pose, size, and shape of the signer's hand. In this letter, Multiscaled Multi-Head Attention Video Transformer Network (MsMHA-VTN)…

Gesture RecognitionHand Gesture RecognitionHand-Gesture Recognition

Hybrid attention network based on progressive embedding scale-context for crowd counting

2021-06-04 · Fusen Wang, Jun Sang, Zhongyuan Wu, Qi Liu 외

The existing crowd counting methods usually adopted attention mechanism to tackle background noise, or applied multi-level features or multi-scales context fusion to tackle scale variation. However, these approaches deal…

Crowd Counting

Mixture of Attention Heads: Selecting Attention Heads Per Token

2022-10-11 · Xiaofeng Zhang, Yikang Shen, Zeyu Huang, Jie zhou 외

Mixture-of-Experts (MoE) networks have been proposed as an efficient way to scale up model capacity and implement conditional computing. However, the study of MoE components mostly focused on the feedforward layer in Tra…

Computational EfficiencyLanguage ModelingLanguage ModellingMachine Translation+2

Learning Hard Retrieval Decoder Attention for Transformers

2020-09-30 · Findings (EMNLP) 2021 11 · Hongfei Xu, Qiuhui Liu, Josef van Genabith, Deyi Xiong

The Transformer translation model is based on the multi-head attention mechanism, which can be parallelized easily. The multi-head attention network performs the scaled dot-product attention function in parallel, empower…

DecoderMachine TranslationRetrievalSentence+1

Attention over Heads: A Multi-Hop Attention for Neural Machine Translation

2019-07-01 · ACL 2019 7 · Shohei Iida, Ryuichiro Kimura, Hongyi Cui, Po-Hsuan Hung 외

In this paper, we propose a multi-hop attention for the Transformer. It refines the attention for an output symbol by integrating that of each head, and consists of two hops. The first hop attention is the scaled dot-pro…

Machine TranslationTranslation