Self-Supervised Transformers for fMRI representation

2021-12-10 · Itzik Malkiel, Gony Rosenman, Lior Wolf, Talma Hendler

We present TFF, which is a Transformer framework for the analysis of functional Magnetic Resonance Imaging (fMRI) data. TFF employs a two-phase training approach. First, self-supervised training is applied to a collection of fMRI scans, where the model is trained to reconstruct 3D volume data. Second, the pre-trained model is fine-tuned on specific tasks, utilizing ground truth labels. Our results show state-of-the-art performance on a variety of fMRI tasks, including age and gender prediction, as well as schizophrenia recognition. Our code for the training, network architecture, and results is attached as supplementary material.

📄 PDF Abstract BibTeX arXiv:2112.05761

Code (2)

gonyrosenman/tff 공식 구현 pytorch

junebeomstics/tff pytorch

Tasks

Gender Prediction

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Self-Supervised Pretraining on Paired Sequences of fMRI Data for Transfer Learning to Brain Decoding Tasks

2023-05-15 · Sean Paulsen, Michael Casey

In this work we introduce a self-supervised pretraining framework for transformers on functional Magnetic Resonance Imaging (fMRI) data. First, we pretrain our architecture on two self-supervised tasks simultaneously to …

Brain Decoding Transfer Learning

Scaling Vision Transformers for Functional MRI with Flat Maps

2025-10-15 · Connor Lane, Mihir Tripathy, Leema Krishna Murali, Ratna Sagari Grandhi 외 arxiv

We study the problem of training self-supervised foundation models for functional MRI. Our main contributions are: (1) we introduce a new model family (CortexMAE) trained using the masked autoencoder framework on 2.1K ho…

GATE: Graph CCA for Temporal SElf-supervised Learning for Label-efficient fMRI Analysis

2022-03-17 · Liang Peng, Nan Wang, Jie Xu, Xiaofeng Zhu 외

In this work, we focus on the challenging task, neuro-disease classification, using functional magnetic resonance imaging (fMRI). In population graph-based disease analysis, graph convolutional neural networks (GCNs) hav…

Classification Representation Learning Self-Supervised Learning

Learning Robust and Task-Invariant Functional Representation from fMRI through Siamese Self-Supervised Learning

2026-05-27 · Jiyao Wang, Peiyu Duan, Nicha C. Dvornek, Lawrence H. Staib 외 arxiv

Functional magnetic resonance imaging (fMRI) is a powerful tool for investigating human brain function. However, the high cost of data acquisition and the inherent subjectivity of psychiatric rating scales often lead to …

Self-Supervised Learning Representation Learning

Sequential Transfer Learning to Decode Heard and Imagined Timbre from fMRI Data

2023-05-22 · Sean Paulsen, Michael Casey

We present a sequential transfer learning framework for transformers on functional Magnetic Resonance Imaging (fMRI) data and demonstrate its significant benefits for decoding musical timbre. In the first of two phases, …

Transfer Learning