Embedding Task Knowledge into 3D Neural Networks via Self-supervised Learning

2020-06-10 · Jiuwen Zhu, Yuexiang Li, Yifan Hu, S. Kevin Zhou

Deep learning highly relies on the amount of annotated data. However, annotating medical images is extremely laborious and expensive. To this end, self-supervised learning (SSL), as a potential solution for deficient annotated data, attracts increasing attentions from the community. However, SSL approaches often design a proxy task that is not necessarily related to target task. In this paper, we propose a novel SSL approach for 3D medical image classification, namely Task-related Contrastive Prediction Coding (TCPC), which embeds task knowledge into training 3D neural networks. The proposed TCPC first locates the initial candidate lesions via supervoxel estimation using simple linear iterative clustering. Then, we extract features from the sub-volume cropped around potential lesion areas, and construct a calibrated contrastive predictive coding scheme for self-supervised learning. Extensive experiments are conducted on public and private datasets. The experimental results demonstrate the effectiveness of embedding lesion-related prior-knowledge into neural networks for 3D medical image classification.

📄 PDF Abstract BibTeX arXiv:2006.05798

Code (0)

등록된 구현이 없습니다.

Tasks

ClusteringGeneral Classification image-classification Image Classification Medical Image Classification Self-Supervised Learning

Methods 이 논문이 사용한 방법론

InfoNCE 설명 없음

Contrastive Predictive Coding Contrastive Predictive Coding (CPC) learns self-supervised representations by predicting the future in latent space by using powerful autoregressive models. The model uses a…

Distilling Knowledge from Self-Supervised Teacher by Embedding Graph Alignment

2022-11-23 · Yuchen Ma, Yanbei Chen, Zeynep Akata

Recent advances have indicated the strengths of self-supervised pre-training for improving representation learning on downstream tasks. Existing works often utilize self-supervised pre-trained models by fine-tuning on do…

Knowledge Distillation Representation Learning Self-Supervised Learning

S-SONDO: Self-Supervised Knowledge Distillation for General Audio Foundation Models

2026-04-27 · Mohammed Ali El Adlouni, Aurian Quelennec, Pierre Chouteau, Geoffroy Peeters 외 arxiv

General audio foundation models have recently achieved remarkable progress, enabling strong performance across diverse tasks. However, state-of-the-art models remain extremely large, often with hundreds of millions of pa…

Knowledge Distillation Model Compression

Stochastic Vision Transformers with Wasserstein Distance-Aware Attention

2023-11-30 · Franciskus Xaverius Erick, Mina Rezaei, Johanna Paula Müller, Bernhard Kainz

Self-supervised learning is one of the most promising approaches to acquiring knowledge from limited labeled data. Despite the substantial advancements made in recent years, self-supervised models have posed a challenge …

Out-of-Distribution Detection Self-Supervised Learning Transfer Learning

Temporal Self-Attention Network for Medical Concept Embedding

2019-09-15 · Xueping Peng, Guodong Long, Tao Shen, Sen Wang 외

In longitudinal electronic health records (EHRs), the event records of a patient are distributed over a long period of time and the temporal relations between the events reflect sufficient domain knowledge to benefit pre…

Clustering

A Theory of Self-Supervised Framework for Few-Shot Learning

2021-01-01 · Zhong Cao, Jiang Lu, Jian Liang, ChangShui Zhang

Recently, self-supervised learning (SSL) algorithms have been applied to Few-shot learning(FSL). FSL aims at distilling transferable knowledge on existing classes with large-scale labeled data to cope with novel classes …

Few-Shot Learning Self-Supervised Learning