paper-with-me

Papers

Publicly Shareable Clinical Large Language Model Built on Synthetic Clinical Notes

2023-09-01 · Sunjun Kweon, Junu Kim, Jiyoun Kim, Sujeong Im, Eunbyeol Cho, Seongsu Bae, JungWoo Oh, Gyubok Lee, Jong Hak Moon, Seng Chan You, Seungjin Baek, Chang Hoon Han, Yoon Bin Jung, Yohan Jo, Edward Choi

The development of large language models tailored for handling patients' clinical notes is often hindered by the limited accessibility and usability of these notes due to strict privacy regulations. To address these challenges, we first create synthetic large-scale clinical notes using publicly available case reports extracted from biomedical literature. We then use these synthetic notes to train our specialized clinical large language model, Asclepius. While Asclepius is trained on synthetic data, we assess its potential performance in real-world applications by evaluating it using real clinical notes. We benchmark Asclepius against several other large language models, including GPT-3.5-turbo and other open-source alternatives. To further validate our approach using synthetic notes, we also compare Asclepius with its variants trained on real clinical notes. Our findings convincingly demonstrate that synthetic clinical notes can serve as viable substitutes for real ones when constructing high-performing clinical language models. This conclusion is supported by detailed evaluations conducted by both GPT-4 and medical professionals. All resources including weights, codes, and data used in the development of Asclepius are made publicly accessible for future research. (https://github.com/starmpcc/Asclepius)

📄 PDF Abstract BibTeX arXiv:2309.00237

Code (1)

starmpcc/asclepius 공식 구현 pytorch

Tasks

Language ModelingLanguage ModellingLarge Language Model

Methods 이 논문이 사용한 방법론

Attention 설명 없음
Cosine Annealing Cosine Annealing is a type of learning rate schedule that has the effect of starting with a large learning rate that is relatively rapidly decreased to a minimum value before…
Position-Wise Feed-Forward Layer 설명 없음
Residual Connection 설명 없음
Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…
Weight Decay 설명 없음
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…
15 Ways to Contact How can i speak to someone at Delta Airlines 설명 없음

Similar Papers 제목 키워드 기반

PARHAF, a human-authored corpus of clinical reports for fictitious patients in French

2026-03-20 · Xavier Tannier, Salam Abbara, Rémi Flicoteaux, Youness Khalil 외 arxiv

The development of clinical natural language processing (NLP) systems is severely hampered by the sensitive nature of medical records, which restricts data sharing under stringent privacy regulations, particularly in Fra…

Towards Automatic Generation of Shareable Synthetic Clinical Notes Using Neural Language Models

2019-05-16 · WS 2019 6 · Oren Melamud, Chaitanya Shivade

Large-scale clinical data is invaluable to driving many computational scientific advances today. However, understandable concerns regarding patient privacy hinder the open dissemination of such data and give rise to subo…

De-identification

Question Answering based Clinical Text Structuring Using Pre-trained Language Model

2019-08-19 · Jiahui Qiu, Yangming Zhou, Zhiyuan Ma, Tong Ruan 외

Clinical text structuring is a critical and fundamental task for clinical research. Traditional methods such as taskspecific end-to-end models and pipeline models usually suffer from the lack of dataset and error propaga…

Language ModelingLanguage ModellingQuestion Answering

GGPONC: A Corpus of German Medical Text with Rich Metadata Based on Clinical Practice Guidelines

2020-07-13 · EMNLP (Louhi) 2020 11 · Florian Borchert, Christina Lohr, Luise Modersohn, Thomas Langer 외

The lack of publicly accessible text corpora is a major obstacle for progress in natural language processing. For medical applications, unfortunately, all language communities other than English are low-resourced. In thi…

Med42-v2: A Suite of Clinical LLMs

2024-08-12 · Clément Christophe, Praveen K Kanithi, Tathagata Raha, Shadab Khan 외

Med42-v2 introduces a suite of clinical large language models (LLMs) designed to address the limitations of generic models in healthcare settings. These models are built on Llama3 architecture and fine-tuned using specia…