paper-with-me

홈 › Papers

Interpretable ICD Code Embeddings with Self- and Mutual-Attention Mechanisms

2019-06-13 · Dixin Luo, Hongteng Xu, Lawrence Carin

We propose a novel and interpretable embedding method to represent the international statistical classification codes of diseases and related health problems (i.e., ICD codes). This method considers a self-attention mechanism within the disease domain and a mutual-attention mechanism jointly between diseases and procedures. This framework captures the clinical relationships between the disease codes and procedures associated with hospital admissions, and it predicts procedures according to diagnosed diseases. A self-attention network is learned to fuse the embeddings of the diseases for each admission. The similarities between the fused disease embedding and the procedure embeddings indicate which procedure should potentially be recommended. Additionally, when learning the embeddings of the ICD codes, the optimal transport between the diseases and the procedures within each admission is calculated as a regularizer of the embeddings. The optimal transport provides a mutual-attention map between diseases and the procedures, which suppresses the ambiguity within their clinical relationships. The proposed method achieves clinically-interpretable embeddings of ICD codes, and outperforms state-of-the-art embedding methods in procedure recommendation.

📄 PDF Abstract BibTeX arXiv:1906.05492

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

On Identifiability in Transformers

2019-08-12 · ICLR 2020 1 · Gino Brunner, Yang Liu, Damián Pascual, Oliver Richter 외

In this paper we delve deep in the Transformer architecture by investigating two of its core components: self-attention and contextual embeddings. In particular, we study the identifiability of attention weights and toke…

Self-attention Presents Low-dimensional Knowledge Graph Embeddings for Link Prediction

2021-12-20 · Peyman Baghershahi, Reshad Hosseini, Hadi Moradi

A few models have tried to tackle the link prediction problem, also known as knowledge graph completion, by embedding knowledge graphs in comparably lower dimensions. However, the state-of-the-art results are attained at…

Knowledge Graph CompletionKnowledge Graph EmbeddingKnowledge Graph EmbeddingsKnowledge Graphs+1

Interpretable Emergent Language Using Inter-Agent Transformers

2025-05-04 · Mannan Bhardwaj

This paper explores the emergence of language in multi-agent reinforcement learning (MARL) using transformers. Existing methods such as RIAL, DIAL, and CommNet enable agent communication but lack interpretability. We pro…

Multi-agent Reinforcement Learning

miCSE: Mutual Information Contrastive Learning for Low-shot Sentence Embeddings

2022-11-09 · Tassilo Klein, Moin Nabi

This paper presents miCSE, a mutual information-based contrastive learning framework that significantly advances the state-of-the-art in few-shot sentence embedding. The proposed approach imposes alignment between the at…

Contrastive LearningFew-Shot LearningSelf-Supervised LearningSentence+3

The Case for Translation-Invariant Self-Attention in Transformer-Based Language Models

2021-06-03 · ACL 2021 5 · Ulme Wennberg, Gustav Eje Henter

Mechanisms for encoding positional information are central for transformer-based language models. In this paper, we analyze the position embeddings of existing language models, finding strong evidence of translation inva…

PositionTranslation