BB-GCN: A Bi-modal Bridged Graph Convolutional Network for Multi-label Chest X-Ray Recognition

2023-02-22 · Guoli Wang, PingPing Wang, Jinyu Cong, Kunmeng Liu, Benzheng Wei

Multi-label chest X-ray (CXR) recognition involves simultaneously diagnosing and identifying multiple labels for different pathologies. Since pathological labels have rich information about their relationship to each other, modeling the co-occurrence dependencies between pathological labels is essential to improve recognition performance. However, previous methods rely on state variable coding and attention mechanisms-oriented to model local label information, and lack learning of global co-occurrence relationships between labels. Furthermore, these methods roughly integrate image features and label embedding, ignoring the alignment and compactness problems in cross-modal vector fusion.To solve these problems, a Bi-modal Bridged Graph Convolutional Network (BB-GCN) model is proposed. This model mainly consists of a backbone module, a pathology Label Co-occurrence relationship Embedding (LCE) module, and a Transformer Bridge Graph (TBG) module. Specifically, the backbone module obtains image visual feature representation. The LCE module utilizes a graph to model the global co-occurrence relationship between multiple labels and employs graph convolutional networks for learning inference. The TBG module bridges the cross-modal vectors more compactly and efficiently through the GroupSum method.We have evaluated the effectiveness of the proposed BB-GCN in two large-scale CXR datasets (ChestX-Ray14 and CheXpert). Our model achieved state-of-the-art performance: the mean AUC scores for the 14 pathologies were 0.835 and 0.813, respectively.The proposed LCE and TBG modules can jointly effectively improve the recognition performance of BB-GCN. Our model also achieves satisfactory results in multi-label chest X-ray recognition and exhibits highly competitive generalization performance.

📄 PDF Abstract BibTeX arXiv:2302.11082

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Adam 설명 없음

Multi-Head Attention 설명 없음

Residual Connection 설명 없음

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Bridged Semantic Alignment for Zero-shot 3D Medical Image Diagnosis

2025-01-07 · Haoran Lai, Zihang Jiang, Qingsong Yao, Rongsheng Wang 외

3D medical images such as Computed tomography (CT) are widely used in clinical practice, offering a great potential for automatic diagnosis. Supervised learning-based approaches have achieved significant progress but rel…

Computed Tomography (CT)Large Language Model Zero-Shot Learning

Equivariance-bridged SO(2)-Invariant Representation Learning using Graph Convolutional Network

2021-06-18 · Sungwon Hwang, Hyungtae Lim, Hyun Myung

Training a Convolutional Neural Network (CNN) to be robust against rotation has mostly been done with data augmentation. In this paper, another progressive vision of research direction is highlighted to encourage less de…

Data Augmentation image-classification Image Classification Representation Learning+1

TAViS: Text-bridged Audio-Visual Segmentation with Foundation Models

2025-06-13 · Ziyang Luo, Nian Liu, Xuguang Yang, Salman Khan 외

Audio-Visual Segmentation (AVS) faces a fundamental challenge of effectively aligning audio and visual modalities. While recent approaches leverage foundation models to address data scarcity, they often rely on single-mo…

cross-modal alignment Segmentation

Bridge to Answer: Structure-aware Graph Interaction Network for Video Question Answering

2021-04-29 · CVPR 2021 1 · Jungin Park, Jiyoung Lee, Kwanghoon Sohn

This paper presents a novel method, termed Bridge to Answer, to infer correct answers for questions about a given video by leveraging adequate graph interactions of heterogeneous crossmodal graphs. To realize this, we le…

Question Answering Video Question Answering

Adaptive label-aware graph convolutional networks for cross-modal retrieval

2021-08-06 · IEEE Transactions on Multimedia 2021 8 · Shengsheng Qian, Dizhan Xue, Quan Fang, Changsheng Xu

The cross-modal retrieval task has raised continuous attention in recent years with the increasing scale of multi-modal data, which has broad application prospects including multimedia data management and intelligent sea…

Cross-Modal Retrieval Representation Learning Retrieval