Transformer Query-Target Knowledge Discovery (TEND): Drug Discovery from CORD-19
Previous work established skip-gram word2vec models could be used to mine knowledge in the materials science literature for the discovery of thermoelectrics. Recent transformer architectures have shown great progress in language modeling and associated fine-tuned tasks, but they have yet to be adapted for drug discovery. We present a RoBERTa transformer-based method that extends the masked language token prediction using query-target conditioning to treat the specificity challenge. The transformer discovery method entails several benefits over the word2vec method including domain-specific (antiviral) analogy performance, negation handling, and flexible query analysis (specific) and is demonstrated on influenza drug discovery. To stimulate COVID-19 research, we release an influenza clinical trials and antiviral analogies dataset used in conjunction with the COVID-19 Open Research Dataset Challenge (CORD-19) literature dataset in the study. We examine k-shot fine-tuning to improve the downstream analogies performance as well as to mine analogies for model explainability. Further, the query-target analysis is verified in a forward chaining analysis against the influenza drug clinical trials dataset, before adapted for COVID-19 drugs (combinations and side-effects) and on-going clinical trials. In consideration of the present topic, we release the model, dataset, and code.
Code (0)
등록된 구현이 없습니다.
Tasks
Drug DiscoveryLanguage ModelingLanguage ModellingNegationSpecificityMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Improving Sequential Query Recommendation with Immediate User Feedback
We propose an algorithm for next query recommendation in interactive data exploration settings, like knowledge discovery for information gathering. The state-of-the-art query recommendation algorithms are based on sequen…
Enhancing Transformers for Generalizable First-Order Logical Entailment
Transformers, as a fundamental deep learning architecture, have demonstrated remarkable capabilities in reasoning. This paper investigates the generalizable first-order logical reasoning ability of transformers with thei…
Logical ReasoningOut-of-Distribution GeneralizationPrompt Engineering for Transformer-based Chemical Similarity Search Identifies Structurally Distinct Functional Analogues
Chemical similarity searches are widely used in-silico methods for identifying new drug-like molecules. These methods have historically relied on structure-based comparisons to compute molecular similarity. Here, we use …
Language ModelingLanguage ModellingPrompt EngineeringMedTQ: Dynamic Topic Discovery and Query Generation for Medical Ontologies
Biomedical ontology refers to a shared conceptualization for a biomedical domain of interest that has vastly improved data management and data sharing through the open data movement. The rapid growth and availability of …
ClusteringManagementImproving Source-Free Target Adaptation with Vision Transformers Leveraging Domain Representation Images
Unsupervised Domain Adaptation (UDA) methods facilitate knowledge transfer from a labeled source domain to an unlabeled target domain, navigating the obstacle of domain shift. While Convolutional Neural Networks (CNNs) a…
Domain AdaptationDomain GeneralizationTransfer LearningUnsupervised Domain Adaptation