HeartBERT: A Self-Supervised ECG Embedding Model for Efficient and Effective Medical Signal Analysis
The HeartBert model is introduced with three primary objectives: reducing the need for labeled data, minimizing computational resources, and simultaneously improving performance in machine learning systems that analyze Electrocardiogram (ECG) signals. Inspired by Bidirectional Encoder Representations from Transformers (BERT) in natural language processing and enhanced with a self-supervised learning approach, the HeartBert model-built on the RoBERTa architecture-generates sophisticated embeddings tailored for ECG-based projects in the medical domain. To demonstrate the versatility, generalizability, and efficiency of the proposed model, two key downstream tasks have been selected: sleep stage detection and heartbeat classification. HeartBERT-based systems, utilizing bidirectional LSTM heads, are designed to address complex challenges. A series of practical experiments have been conducted to demonstrate the superiority and advancements of HeartBERT, particularly in terms of its ability to perform well with smaller training datasets, reduced learning parameters, and effective performance compared to rival models. The code and data are publicly available at https://github.com/ecgResearch/HeartBert.
Code (1)
Tasks
Heartbeat ClassificationSelf-Supervised LearningSleep Stage DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Temporal Self-Attention Network for Medical Concept Embedding
In longitudinal electronic health records (EHRs), the event records of a patient are distributed over a long period of time and the temporal relations between the events reflect sufficient domain knowledge to benefit pre…
ClusteringTowards Fair Medical AI: Adversarial Debiasing of 3D CT Foundation Embeddings
Self-supervised learning has revolutionized medical imaging by enabling efficient and generalizable feature extraction from large-scale unlabeled datasets. Recently, self-supervised foundation models have been extended t…
Computed Tomography (CT)FairnessSelf-Supervised LearningExploiting Sentence Embedding for Medical Question Answering
Despite the great success of word embedding, sentence embedding remains a not-well-solved problem. In this paper, we present a supervised learning framework to exploit sentence embedding for the medical question answerin…
Medical Question AnsweringQuestion AnsweringSentenceSentence Embedding+1Embedding Task Knowledge into 3D Neural Networks via Self-supervised Learning
Deep learning highly relies on the amount of annotated data. However, annotating medical images is extremely laborious and expensive. To this end, self-supervised learning (SSL), as a potential solution for deficient ann…
ClusteringGeneral Classificationimage-classificationImage Classification+2MLVICX: Multi-Level Variance-Covariance Exploration for Chest X-ray Self-Supervised Representation Learning
Self-supervised learning (SSL) is potentially useful in reducing the need for manual annotation and making deep learning models accessible for medical image analysis tasks. By leveraging the representations learned from …
Medical DiagnosisMedical Image AnalysisRepresentation LearningSelf-Supervised Learning