Learning Conceptual-Contextual Embeddings for Medical Text
External knowledge is often useful for natural language understanding tasks. We introduce a contextual text representation model called Conceptual-Contextual (CC) embeddings, which incorporates structured knowledge into text representations. Unlike entity embedding methods, our approach encodes a knowledge graph into a context model. CC embeddings can be easily reused for a wide range of tasks just like pre-trained language models. Our model effectively encodes the huge UMLS database by leveraging semantic generalizability. Experiments on electronic health records (EHRs) and medical text processing benchmarks showed our model gives a major boost to the performance of supervised medical NLP tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Natural Language UnderstandingSimilar Papers 제목 키워드 기반
Evaluating the Underlying Gender Bias in Contextualized Word Embeddings
Gender bias is highly impacting natural language processing applications. Word embeddings have clearly been proven both to keep and amplify gender biases that are present in current data sources. Recently, contextualized…
SentenceWord EmbeddingsGender bias in (non)-contextual clinical word embeddings for stereotypical medical categories
Clinical word embeddings are extensively used in various Bio-NLP problems as a state-of-the-art feature vector representation. Although they are quite successful at the semantic representation of words, due to the datase…
Word EmbeddingsBERT's Conceptual Cartography: Mapping the Landscapes of Meaning
Conceptual Engineers want to make words better. However, they often underestimate how varied our usage of words is. In this paper, we take the first steps in exploring the contextual nuances of words by creating conceptu…
Word EmbeddingsWhen Specialization Helps: Using Pooled Contextualized Embeddings to Detect Chemical and Biomedical Entities in Spanish
The recognition of pharmacological substances, compounds and proteins is an essential preliminary work for the recognition of relations between chemicals and other biomedically relevant units. In this paper, we describe …
ArticlesWord EmbeddingsOn Learning Better Embeddings from Chinese Clinical Records: Study on Combining In-Domain and Out-Domain Data
High quality word embeddings are of great significance to advance applications of biomedical natural language processing. In recent years, a surge of interest on how to learn good embeddings and evaluate embedding qualit…
Disease PredictionInformation RetrievalLanguage ModelingLanguage Modelling+1