Fluoroformer: Scaling multiple instance learning to multiplexed images via attention-based channel fusion
Though multiple instance learning (MIL) has been a foundational strategy in computational pathology for processing whole slide images (WSIs), current approaches are designed for traditional hematoxylin and eosin (H&E) slides rather than emerging multiplexed technologies. Here, we present an MIL strategy, the Fluoroformer module, that is specifically tailored to multiplexed WSIs by leveraging scaled dot-product attention (SDPA) to interpretably fuse information across disparate channels. On a cohort of 434 non-small cell lung cancer (NSCLC) samples, we show that the Fluoroformer both obtains strong prognostic performance and recapitulates immuno-oncological hallmarks of NSCLC. Our technique thereby provides a path for adapting state-of-the-art AI techniques to emerging spatial biology assays.
Code (1)
Tasks
Multiple Instance Learningwhole slide imagesMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MITI Minimum Information guidelines for highly multiplexed tissue images
The imminent release of tissue atlases combining multi-channel microscopy with single cell sequencing and other omics data from normal and diseased specimens creates an urgent need for data and metadata standards that gu…
Semi-supervised variational autoencoder for cell feature extraction in multiplexed immunofluorescence images
Advancements in digital imaging technologies have sparked increased interest in using multiplexed immunofluorescence (mIF) images to visualise and identify the interactions between specific immunophenotypes with the tumo…
Phenotype classificationPhysics-Free Spectrally Multiplexed Photometric Stereo under Unknown Spectral Composition
In this paper, we present a groundbreaking spectrally multiplexed photometric stereo approach for recovering surface normals of dynamic surfaces without the need for calibrated lighting or sensors, a notable advancement …
A SSIM Guided cGAN Architecture For Clinically Driven Generative Image Synthesis of Multiplexed Spatial Proteomics Channels
Here we present a structural similarity index measure (SSIM) guided conditional Generative Adversarial Network (cGAN) that generatively performs image-to-image (i2i) synthesis to generate photo-accurate protein channels …
EthicsGenerative Adversarial NetworkImage GenerationSSIMDeep learning-enabled virtual multiplexed immunostaining of label-free tissue for vascular invasion assessment
Immunohistochemistry (IHC) has transformed clinical pathology by enabling the visualization of specific proteins within tissue sections. However, traditional IHC requires one tissue section per stain, exhibits section-to…