paper-with-me

홈 › Papers

KG-MTT-BERT: Knowledge Graph Enhanced BERT for Multi-Type Medical Text Classification

2022-10-08 · Yong He, Cheng Wang, Shun Zhang, Nan Li, Zhaorong Li, Zhenyu Zeng

Medical text learning has recently emerged as a promising area to improve healthcare due to the wide adoption of electronic health record (EHR) systems. The complexity of the medical text such as diverse length, mixed text types, and full of medical jargon, poses a great challenge for developing effective deep learning models. BERT has presented state-of-the-art results in many NLP tasks, such as text classification and question answering. However, the standalone BERT model cannot deal with the complexity of the medical text, especially the lengthy clinical notes. Herein, we develop a new model called KG-MTT-BERT (Knowledge Graph Enhanced Multi-Type Text BERT) by extending the BERT model for long and multi-type text with the integration of the medical knowledge graph. Our model can outperform all baselines and other state-of-the-art models in diagnosis-related group (DRG) classification, which requires comprehensive medical text for accurate classification. We also demonstrated that our model can effectively handle multi-type text and the integration of medical knowledge graph can significantly improve the performance.

📄 PDF Abstract BibTeX arXiv:2210.03970

Code (0)

등록된 구현이 없습니다.

Tasks

ClassificationQuestion Answeringtext-classificationText Classification

Methods 이 논문이 사용한 방법론

Refunds@Expedia|||How do I get a full refund from Expedia? “How do I get a full refund from Expedia? How do I get a full refund from Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Quick Help &…
Multi-Head Attention 설명 없음
Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
Residual Connection 설명 없음
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…
Weight Decay 설명 없음

Similar Papers 제목 키워드 기반

ExBERT: An External Knowledge Enhanced BERT for Natural Language Inference

2021-08-03 · Amit Gajbhiye, Noura Al Moubayed, Steven Bradley

Neural language representation models such as BERT, pre-trained on large-scale unstructured corpora lack explicit grounding to real-world commonsense knowledge and are often unable to remember facts required for reasonin…

Knowledge GraphsNatural Language Inference

SEBERTNets: Sequence Enhanced BERT Networks for Event Entity Extraction Tasks Oriented to the Finance Field

2024-01-21 · Congqing He, Xiangyu Zhu, Yuquan Le, Yuzhong Liu 외

Event extraction lies at the cores of investment analysis and asset management in the financial field, and thus has received much attention. The 2019 China conference on knowledge graph and semantic computing (CCKS) chal…

Asset ManagementEvent ExtractionManagement

Multiclass Hate Speech Detection with RoBERTa-OTA: Integrating Transformer Attention and Graph Convolutional Networks

2026-02-03 · Mahmoud Abusaqer, Jamil Saquer arxiv

Multiclass hate speech detection across demographic categories remains computationally challenging due to implicit targeting strategies and linguistic variability in social media content. Existing approaches rely solely …

Computational EfficiencyHate Speech Detection

Revisiting and Advancing Chinese Natural Language Understanding with Accelerated Heterogeneous Knowledge Pre-training

2022-10-11 · Taolin Zhang, Junwei DOng, Jianing Wang, Chengyu Wang 외

Recently, knowledge-enhanced pre-trained language models (KEPLMs) improve context-aware representations via learning from structured relations in knowledge graphs, and/or linguistic knowledge from syntactic or dependency…

GPUKnowledge GraphsLanguage ModelingLanguage Modelling+2

Knowledge Enhanced Embedding: Improve Model Generalization Through Knowledge Graphs

2021-11-16 · ACL ARR November 2021 11 · Anonymous

Pre-trained language models have achieved excellent results in NLP and NLI, and since the birth of Bert, various new types of Bert have emerged.They are able to grasp the ubiquitous linguistic representational informatio…

Knowledge GraphsSentence