홈 › Papers

Transfer learning applied to text classification in Spanish radiological reports

2020-05-01 · LREC 2020 5 · Pilar L{\'o}pez {\'U}beda, Manuel Carlos D{\'\i}az-Galiano, L. Alfonso Urena Lopez, Maite Martin, Teodoro Mart{\'\i}n-Noguerol, Antonio Luna

Pre-trained text encoders have rapidly advanced the state-of-the-art on many Natural Language Processing tasks. This paper presents the use of transfer learning methods applied to the automatic detection of codes in radiological reports in Spanish. Assigning codes to a clinical document is a popular task in NLP and in the biomedical domain. These codes can be of two types: standard classifications (e.g. ICD-10) or specific to each clinic or hospital. In this study we show a system using specific radiology clinic codes. The dataset is composed of 208,167 radiology reports labeled with 89 different codes. The corpus has been evaluated with three methods using the BERT model applied to Spanish: Multilingual BERT, BETO and XLM. The results are interesting obtaining 70{\%} of F1-score with a pre-trained multilingual model.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Classification General Classification text-classification Text Classification Transfer Learning

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Weight Decay 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Attention 설명 없음

The Utility of General Domain Transfer Learning for Medical Language Tasks

2020-02-16 · Daniel Ranti, Katie Hanss, Shan Zhao, Varun Arvind 외

The purpose of this study is to analyze the efficacy of transfer learning techniques and transformer-based models as applied to medical natural language processing (NLP) tasks, specifically radiological text classificati…

Classification General Classification Multi-class Classification regression+3

NLP-CIC-WFU at SocialDisNER: Disease Mention Extraction in Spanish Tweets Using Transfer Learning and Search by Propagation

2022-10-01 · SMM4H (COLING) 2022 10 · Antonio Tamayo, Alexander Gelbukh, Diego Burgos

Named entity recognition (e.g., disease mention extraction) is one of the most relevant tasks for data mining in the medical field. Although it is a well-known challenge, the bulk of the efforts to tackle this task have …

named-entity-recognition Named Entity Recognition Named Entity Recognition (NER)Transfer Learning

OmniRad: A Radiological Foundation Model for Multi-Task Medical Image Analysis

2026-02-04 · Luca Zedda, Andrea Loddo, Cecilia Di Ruberto arxiv

Radiological analysis increasingly benefits from pretrained visual representations that can support heterogeneous downstream tasks across imaging modalities. In this work, we introduce OmniRad, a self-supervised radiolog…

Punctuation Restoration in Spanish Customer Support Transcripts using Transfer Learning

2022-05-27 · DeepLo 2022 7 · Xiliang Zhu, Shayna Gardiner, David Rossouw, Tere Roldán 외

Automatic Speech Recognition (ASR) systems typically produce unpunctuated transcripts that have poor readability. In addition, building a punctuation restoration system is challenging for low-resource languages, especial…

Automatic Speech Recognition Automatic Speech Recognition (ASR)Cross-Lingual Transfer Domain Adaptation+4

BIMCV COVID-19+: a large annotated dataset of RX and CT images from COVID-19 patients

2020-06-01 · Maria de la Iglesia Vayá, Jose Manuel Saborit, Joaquim Angel Montell, Antonio Pertusa 외

This paper describes BIMCV COVID-19+, a large dataset from the Valencian Region Medical ImageBank (BIMCV) containing chest X-ray images CXR (CR, DX) and computed tomography (CT) imaging of COVID-19+ patients along with t…

Computed Tomography (CT)Diagnostic Semantic Segmentation