Prediction of ICD Codes with Clinical BERT Embeddings and Text Augmentation with Label Balancing using MIMIC-III

2020-08-24 · Brent Biseda, Gaurav Desai, Haifeng Lin, Anish Philip

This paper achieves state of the art results for the ICD code prediction task using the MIMIC-III dataset. This was achieved through the use of Clinical BERT (Alsentzer et al., 2019). embeddings and text augmentation and label balancing to improve F1 scores for both ICD Chapter as well as ICD disease codes. We attribute the improved performance mainly to the use of novel text augmentation to shuffle the order of sentences during training. In comparison to the Top-32 ICD code prediction (Keyang Xu, et. al.) with an F1 score of 0.76, we achieve a final F1 score of 0.75 but on a total of the top 50 ICD codes.

📄 PDF Abstract BibTeX arXiv:2008.10492

Code (0)

등록된 구현이 없습니다.

Tasks

AttributePredictionText Augmentation

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

Weight Decay 설명 없음

Adam 설명 없음

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

WordPiece 설명 없음

Multi-Head Attention 설명 없음

Residual Connection 설명 없음

A Multi-View Joint Learning Framework for Embedding Clinical Codes and Text Using Graph Neural Networks

2023-01-27 · Lecheng Kong, Christopher King, Bradley Fritz, Yixin Chen

Learning to represent free text is a core task in many clinical machine learning (ML) applications, as clinical text contains observations and plans not otherwise available for inference. State-of-the-art methods use lar…

Graph Neural Network MULTI-VIEW LEARNING

CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction tasks

2021-11-10 · Chao Pang, Xinzhuo Jiang, Krishna S Kalluri, Matthew Spotnitz 외

Embedding algorithms are increasingly used to represent clinical concepts in healthcare for improving machine learning tasks such as clinical phenotyping and disease prediction. Recent studies have adapted state-of-the-a…

Disease Prediction Transfer Learning

Automatic Transformation of Clinical Narratives into Structured Format

2021-09-01 · RANLP 2021 9 · Sylvia Vassileva, Gergana Todorova, Kristina Ivanova, Boris Velichkov 외

Vast amounts of data in healthcare are available in unstructured text format, usually in the local language of the countries. These documents contain valuable information. Secondary use of clinical narratives and informa…

Binary Classification Negation Word Embeddings

Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings

2020-03-11 · Haoran Zhang, Amy X. Lu, Mohamed Abdalla, Matthew McDermott 외

In this work, we examine the extent to which embeddings may encode marginalized populations differently, and how this may lead to a perpetuation of biases and worsened performance on clinical tasks. We pretrain deep embe…

Fairness Word Embeddings

Attention Networks for Augmenting Clinical Text with Support Sets for Diagnosis Prediction

2022-10-01 · COLING 2022 10 · Paul Grundmann, Tom Oberhauser, Felix Gers, Alexander Löser

Diagnosis prediction on admission notes is a core clinical task. However, these notes may incompletely describe the patient. Also, clinical language models may suffer from idiosyncratic language or imbalanced vocabulary …

Diagnostic Prediction