MolE: a molecular foundation model for drug discovery
Models that accurately predict properties based on chemical structure are valuable tools in drug discovery. However, for many properties, public and private training sets are typically small, and it is difficult for the models to generalize well outside of the training data. Recently, large language models have addressed this problem by using self-supervised pretraining on large unlabeled datasets, followed by fine-tuning on smaller, labeled datasets. In this paper, we report MolE, a molecular foundation model that adapts the DeBERTa architecture to be used on molecular graphs together with a two-step pretraining strategy. The first step of pretraining is a self-supervised approach focused on learning chemical structures, and the second step is a massive multi-task approach to learn biological information. We show that fine-tuning pretrained MolE achieves state-of-the-art results on 9 of the 22 ADMET tasks included in the Therapeutic Data Commons.
Code (0)
등록된 구현이 없습니다.
Tasks
Drug DiscoverymodelMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discovery
The rapid evolution of artificial intelligence in drug discovery encounters challenges with generalization and extensive training, yet Large Language Models (LLMs) offer promise in reshaping interactions with complex mol…
Drug DiscoveryMolecule CaptioningMMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery
General-purpose large language models (LLMs) that rely on in-context learning do not reliably deliver the scientific understanding and performance required for drug discovery tasks. Simply increasing model size or introd…
Activity PredictionDrug DiscoveryFolding, Reasoning, and Scaling with Open-source Drug Discovery Engine
Accurately modeling biomolecular interactions is a central bottleneck in biology and therapeutic discovery. Here, we introduce Open Drug Discovery Engine (OpenDDE), an open-source, all-atom biomolecular foundation model …
Drug DiscoveryGraph Neural Networks in Modern AI-aided Drug Discovery
Graph neural networks (GNNs), as topology/structure-aware models within deep learning, have emerged as powerful tools for AI-aided drug discovery (AIDD). By directly operating on molecular graphs, GNNs offer an intuitive…
Drug Discoverygraph constructionMeta-LearningMolecular Property Prediction+4An effective self-supervised framework for learning expressive molecular global representations to drug discovery
How to produce expressive molecular representations is a fundamental challenge in artificial intelligence-driven drug discovery. Graph neural network (GNN) has emerged as a powerful technique for modeling molecular data.…
Drug DiscoveryGraph Neural Network