paper-with-me

홈 › Papers

Polyhedra Encoding Transformers: Enhancing Diffusion MRI Analysis Beyond Voxel and Volumetric Embedding

2025-01-23 · Tianyuan Yao, Zhiyuan Li, Praitayini Kanakaraj, Derek B. Archer, Kurt Schilling, Lori Beason-Held, Susan Resnick, Bennett A. Landman, Yuankai Huo

Diffusion-weighted Magnetic Resonance Imaging (dMRI) is an essential tool in neuroimaging. It is arguably the sole noninvasive technique for examining the microstructural properties and structural connectivity of the brain. Recent years have seen the emergence of machine learning and data-driven approaches that enhance the speed, accuracy, and consistency of dMRI data analysis. However, traditional deep learning models often fell short, as they typically utilize pixel-level or volumetric patch-level embeddings similar to those used in structural MRI, and do not account for the unique distribution of various gradient encodings. In this paper, we propose a novel method called Polyhedra Encoding Transformer (PE-Transformer) for dMRI, designed specifically to handle spherical signals. Our approach involves projecting an icosahedral polygon onto a unit sphere to resample signals from predetermined directions. These resampled signals are then transformed into embeddings, which are processed by a transformer encoder that incorporates orientational information reflective of the icosahedral structure. Through experimental validation with various gradient encoding protocols, our method demonstrates superior accuracy in estimating multi-compartment models and Fiber Orientation Distributions (FOD), outperforming both conventional CNN architectures and standard transformers.

📄 PDF Abstract BibTeX arXiv:2501.13352

Code (0)

등록된 구현이 없습니다.

Tasks

Diffusion MRI

Methods 이 논문이 사용한 방법론

Attention 설명 없음
Adam 설명 없음
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…
Residual Connection 설명 없음
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Similar Papers 제목 키워드 기반

LEDiT: Your Length-Extrapolatable Diffusion Transformer without Positional Encoding

2025-03-06 · Shen Zhang, Yaning Tan, Siyuan Liang, Linze Li 외

Diffusion transformers(DiTs) struggle to generate images at resolutions higher than their training resolutions. The primary obstacle is that the explicit positional encodings(PE), such as RoPE, need extrapolation which d…

Expressivity of Transformers: A Tropical Geometry Perspective

2026-04-16 · Ye Su, Yong Liu arxiv

To quantify the geometric expressivity of transformers, we introduce a tropical geometry framework to characterize their exact spatial partitioning capabilities. By modeling self-attention as a vector-valued tropical rat…

CoD-Lite: Real-Time Diffusion-Based Generative Image Compression

2026-04-14 · Zhaoyang Jia, Naifu Xue, Zihan Zheng, Jiahao Li 외 arxiv

Recent advanced diffusion methods typically derive strong generative priors by scaling diffusion transformers. However, scaling fails to generalize when adapted for real-time compression scenarios that demand lightweight…

Image Compression

Enhancing Graph Transformers with Hierarchical Distance Structural Encoding

2023-08-22 · Yuankai Luo, Hongkang Li, Lei Shi, Xiao-Ming Wu

Graph transformers need strong inductive biases to derive meaningful attention scores. Yet, current methods often fall short in capturing longer ranges, hierarchical structures, or community structures, which are common …

Graph ClassificationGraph RegressionNode Classification

Contextual Counting: A Mechanistic Study of Transformers on a Quantitative Task

2024-05-30 · Siavash Golkar, Alberto Bietti, Mariel Pettee, Michael Eickenberg 외

Transformers have revolutionized machine learning across diverse domains, yet understanding their behavior remains crucial, particularly in high-stakes applications. This paper introduces the contextual counting task, a …

object-detectionObject Detection