Pre-training Transformers for Molecular Property Prediction Using Reaction Prediction

2022-07-06 · Johan Broberg, Maria Bånkestad, Erik Ylipää

Molecular property prediction is essential in chemistry, especially for drug discovery applications. However, available molecular property data is often limited, encouraging the transfer of information from related data. Transfer learning has had a tremendous impact in fields like Computer Vision and Natural Language Processing signaling for its potential in molecular property prediction. We present a pre-training procedure for molecular representation learning using reaction data and use it to pre-train a SMILES Transformer. We fine-tune and evaluate the pre-trained model on 12 molecular property prediction tasks from MoleculeNet within physical chemistry, biophysics, and physiology and show a statistically significant positive effect on 5 of the 12 tasks compared to a non-pre-trained baseline model.

📄 PDF Abstract BibTeX arXiv:2207.02724

Code (0)

등록된 구현이 없습니다.

Tasks

Drug Discovery Molecular Property Predictionmolecular representationPredictionProperty PredictionRepresentation Learning Transfer Learning

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Multi-Head Attention 설명 없음

Residual Connection 설명 없음

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Position-Wise Feed-Forward Layer 설명 없음

Adam 설명 없음

MolKD: Distilling Cross-Modal Knowledge in Chemical Reactions for Molecular Property Prediction

2023-05-03 · Liang Zeng, Lanqing Li, Jian Li

How to effectively represent molecules is a long-standing challenge for molecular property prediction and drug discovery. This paper studies this problem and proposes to incorporate chemical domain knowledge, specificall…

Drug Discovery Molecular Property Prediction Prediction Property Prediction

ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction

2020-10-19 · Seyone Chithrananda, Gabriel Grand, Bharath Ramsundar

GNNs and chemical fingerprints are the predominant approaches to representing molecules for property prediction. However, in NLP, transformers have become the de-facto standard for representation learning thanks to their…

Molecular Property Prediction molecular representation Prediction Property Prediction+1

Mol-LLM: Multimodal Generalist Molecular LLM with Improved Graph Utilization

2025-02-05 · Chanhui Lee, Hanbum Ko, Yuheon Song, Yongjun Jeong 외

Recent advances in large language models (LLMs) have led to models that tackle diverse molecular tasks, such as chemical reaction prediction and molecular property prediction. Large-scale molecular instruction-tuning dat…

Chemical Reaction Prediction Molecular Property Prediction Molecule Captioning Prediction+1

Triplet Interaction Improves Graph Transformers: Accurate Molecular Graph Learning with Triplet Graph Transformers

2024-02-07 · Md Shamim Hussain, Mohammed J. Zaki, Dharmashankar Subramanian

Graph transformers typically lack third-order interactions, limiting their geometric understanding which is crucial for tasks like molecular geometry prediction. We propose the Triplet Graph Transformer (TGT) that enable…

Drug Discovery Graph Learning Graph Property Prediction Graph Regression+8

Uncertainty quantification of molecular property prediction using Bayesian neural network models

2018-11-19 · Seongok Ryu, Yongchan Kwon, Woo Youn Kim

In chemistry, deep neural network models have been increasingly utilized in a variety of applications such as molecular property predictions, novel molecule designs, and planning chemical reactions. Despite the rapid inc…

Molecular Property Prediction Property Prediction Uncertainty Quantification