paper-with-me

홈 › Papers

MTS-Net: Dual-Enhanced Positional Multi-Head Self-Attention for 3D CT Diagnosis of May-Thurner Syndrome

2024-06-07 · Yixin Huang, Yiqi Jin, Ke Tao, Kaijian Xia, Jianfeng Gu, Lei Yu, Lan Du, Cunjian Chen

May-Thurner Syndrome (MTS), also known as iliac vein compression syndrome or Cockett's syndrome, is a condition potentially impacting over 20 percent of the population, leading to an increased risk of iliofemoral deep venous thrombosis. In this paper, we present a 3D-based deep learning approach called MTS-Net for diagnosing May-Thurner Syndrome using CT scans. To effectively capture the spatial-temporal relationship among CT scans and emulate the clinical process of diagnosing MTS, we propose a novel attention module called the dual-enhanced positional multi-head self-attention (DEP-MHSA). The proposed DEP-MHSA reconsiders the role of positional embedding and incorporates a dual-enhanced positional embedding in both attention weights and residual connections. Further, we establish a new dataset, termed MTS-CT, consisting of 747 subjects. Experimental results demonstrate that our proposed approach achieves state-of-the-art MTS diagnosis results, and our self-attention design facilitates the spatial-temporal modeling. We believe that our DEP-MHSA is more suitable to handle CT image sequence modeling and the proposed dataset enables future research on MTS diagnosis. We make our code and dataset publicly available at: https://github.com/Nutingnon/MTS_dep_mhsa.

📄 PDF Abstract BibTeX arXiv:2406.04680

Code (1)

nutingnon/mts_dep_mhsa 공식 구현 pytorch

Methods 이 논문이 사용한 방법론

MTS 설명 없음

Similar Papers 제목 키워드 기반

Dual Triangle Attention: Effective Bidirectional Attention Without Positional Embeddings

2026-04-09 · Logan Hallee, Jason P. Gleghorn arxiv

Bidirectional transformers are the foundation of many sequence modeling tasks across natural, biological, and chemical language domains, but they are permutation-invariant without explicit positional embeddings. In contr…

Enlivening Redundant Heads in Multi-head Self-attention for Machine Translation

2021-11-01 · EMNLP 2021 11 · Tianfu Zhang, Heyan Huang, Chong Feng, Longbing Cao

Multi-head self-attention recently attracts enormous interest owing to its specialized functions, significant parallelizable computation, and flexible extensibility. However, very recent empirical studies show that some …

Machine TranslationTranslation

Enhanced Transformer Architecture for Natural Language Processing

2023-10-17 · Woohyeon Moon, TaeYoung Kim, Bumgeun Park, Dongsoo Har

Transformer is a state-of-the-art model in the field of natural language processing (NLP). Current NLP models primarily increase the number of transformers to improve processing performance. However, this technique requi…

Translation

Self-Training for Compositional Neural NLG in Task-Oriented Dialogue

2021-08-01 · INLG (ACL) 2021 8 · Xintong Li, Symon Stevens-Guille, Aleksandre Maskharashvili, Michael White

Neural approaches to natural language generation in task-oriented dialogue have typically required large amounts of annotated training data to achieve satisfactory performance, especially when generating from composition…

Text Generation

ComplexFormer: Disruptively Advancing Transformer Inference Ability via Head-Specific Complex Vector Attention

2025-05-15 · Jintian Shao, Hongyi Huang, Jiayi Wu, Beiwen Zhang 외

Transformer models rely on self-attention to capture token dependencies but face challenges in effectively integrating positional information while allowing multi-head attention (MHA) flexibility. Prior methods often mod…

Code GenerationLanguage ModelingLanguage ModellingMathematical Reasoning+1