Adversarial Uni- and Multi-modal Stream Networks for Multimodal Image Registration

2020-07-06 · Zhe Xu, Jie Luo, Jiangpeng Yan, Ritvik Pulya, Xiu Li, William Wells III, Jayender Jagadeesan

Deformable image registration between Computed Tomography (CT) images and Magnetic Resonance (MR) imaging is essential for many image-guided therapies. In this paper, we propose a novel translation-based unsupervised deformable image registration method. Distinct from other translation-based methods that attempt to convert the multimodal problem (e.g., CT-to-MR) into a unimodal problem (e.g., MR-to-MR) via image-to-image translation, our method leverages the deformation fields estimated from both: (i) the translated MR image and (ii) the original CT image in a dual-stream fashion, and automatically learns how to fuse them to achieve better registration performance. The multimodal registration network can be effectively trained by computationally efficient similarity metrics without any ground-truth deformation. Our method has been evaluated on two clinical datasets and demonstrates promising results compared to state-of-the-art traditional and learning-based methods.

📄 PDF Abstract BibTeX arXiv:2007.02790

Code (0)

등록된 구현이 없습니다.

Tasks

Computed Tomography (CT)Image Registration Image-to-Image Translation Translation

Revisiting the Adversarial Robustness of Vision Language Models: a Multimodal Perspective

2024-04-30 · Wanqi Zhou, Shuanghao Bai, Danilo P. Mandic, Qibin Zhao 외

Pretrained vision-language models (VLMs) like CLIP exhibit exceptional generalization across diverse downstream tasks. While recent studies reveal their vulnerability to adversarial attacks, research to date has primaril…

Adversarial Defense Adversarial Robustness Adversarial Text

Unified Attentional Generative Adversarial Network for Brain Tumor Segmentation From Multimodal Unpaired Images

2019-07-08 · Wenguang Yuan, Jia Wei, Jiabing Wang, Qianli Ma 외

In medical applications, the same anatomical structures may be observed in multiple modalities despite the different image characteristics. Currently, most deep models for multimodal segmentation rely on paired registere…

Brain Tumor Segmentation Generative Adversarial Network Segmentation Translation+1

AdvCLIP: Downstream-agnostic Adversarial Examples in Multimodal Contrastive Learning

2023-08-14 · Ziqi Zhou, Shengshan Hu, Minghui Li, Hangtao Zhang 외

Multimodal contrastive learning aims to train a general-purpose feature extractor, such as CLIP, on vast amounts of raw, unlabeled paired image-text data. This can greatly benefit various complex downstream tasks, includ…

Contrastive Learning Generative Adversarial Network image-classification Image Classification+2

Enhancing Adversarial Transferability in Visual-Language Pre-training Models via Local Shuffle and Sample-based Attack

2025-11-02 · Xin Liu, Aoyang Zhou, Aoyang Zhou arxiv

Visual-Language Pre-training (VLP) models have achieved significant performance across various downstream tasks. However, they remain vulnerable to adversarial examples. While prior efforts focus on improving the adversa…

VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Models

2023-10-07 · NeurIPS 2023 11 · Ziyi Yin, Muchao Ye, Tianrong Zhang, Tianyu Du 외

Vision-Language (VL) pre-trained models have shown their superiority on many multimodal tasks. However, the adversarial robustness of such models has not been fully explored. Existing approaches mainly focus on exploring…

Adversarial Robustness