홈 › Papers

Patch-Mix Contrastive Learning with Audio Spectrogram Transformer on Respiratory Sound Classification

2023-05-23 · Sangmin Bae, June-Woo Kim, Won-Yang Cho, Hyerim Baek, Soyoun Son, Byungjo Lee, Changwan Ha, Kyongpil Tae, Sungnyun Kim, Se-Young Yun

Respiratory sound contains crucial information for the early diagnosis of fatal lung diseases. Since the COVID-19 pandemic, there has been a growing interest in contact-free medical care based on electronic stethoscopes. To this end, cutting-edge deep learning models have been developed to diagnose lung diseases; however, it is still challenging due to the scarcity of medical data. In this study, we demonstrate that the pretrained model on large-scale visual and audio datasets can be generalized to the respiratory sound classification task. In addition, we introduce a straightforward Patch-Mix augmentation, which randomly mixes patches between different samples, with Audio Spectrogram Transformer (AST). We further propose a novel and effective Patch-Mix Contrastive Learning to distinguish the mixed representations in the latent space. Our method achieves state-of-the-art performance on the ICBHI dataset, outperforming the prior leading score by an improvement of 4.08%.

📄 PDF Abstract BibTeX arXiv:2305.14032

Code (1)

raymin0223/patch-mix_contrastive_learning 공식 구현 pytorch

Tasks

Audio Classification Contrastive LearningSound Classification

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Multi-View Spectrogram Transformer for Respiratory Sound Classification

2023-11-16 · Wentao He, Yuchen Yan, Jianfeng Ren, Ruibin Bai 외

Deep neural networks have been applied to audio spectrograms for respiratory sound classification. Existing models often treat the spectrogram as a synthetic image while overlooking its physical characteristics. In this …

Classification Sound Classification

Lung-SRAD: Spectral-Aware Regularized Audio DASS with Dual-Axis Patch-Mix Contrastive Learning for Respiratory Sound Classification

2026-06-10 · Hemansh Shridhar, Miika Toikkanen, June-Woo Kim arxiv

Recent respiratory sound classification (RSC) studies largely rely on CLS-token driven self-attention architectures such as the Audio Spectrogram Transformer (AST). While effective at modeling global context, recent anal…

Representation Learning Contrastive Learning

MAST: Multiscale Audio Spectrogram Transformers

2022-11-02 · Sreyan Ghosh, Ashish Seth, S. Umesh, Dinesh Manocha

We present Multiscale Audio Spectrogram Transformer (MAST) for audio classification, which brings the concept of multiscale feature hierarchies to the Audio Spectrogram Transformer (AST). Given an input audio spectrogram…

Audio Classification Keyword Spotting Self-Supervised Learning

Abnormal Respiratory Sound Identification Using Audio-Spectrogram Vision Transformer

2024-05-14 · Whenty Ariyanti, Kai-Chun Liu, Kuan-Yu Chen, Yu Tsao

Respiratory disease, the third leading cause of deaths globally, is considered a high-priority ailment requiring significant research on identification and treatment. Stethoscope-recorded lung sounds and artificial intel…

Classification of Adventitious Sounds Combining Cochleogram and Vision Transformers

2024-11-08 · Loredana Daria Mang, Francisco David Gonzalez Martinez, Damian Martinez Munoz, Sebastian Garcia Galan 외

Early identification of respiratory irregularities is critical for improving lung health and reducing global mortality rates. The analysis of respiratory sounds plays a significant role in characterizing the respiratory …

Audio Classification Classification Sound Classification