GeomCLIP: Contrastive Geometry-Text Pre-training for Molecules

2024-11-16 · Teng Xiao, Chao Cui, Huaisheng Zhu, Vasant G. Honavar

Pretraining molecular representations is crucial for drug and material discovery. Recent methods focus on learning representations from geometric structures, effectively capturing 3D position information. Yet, they overlook the rich information in biomedical texts, which detail molecules' properties and substructures. With this in mind, we set up a data collection effort for 200K pairs of ground-state geometric structures and biomedical texts, resulting in a PubChem3D dataset. Based on this dataset, we propose the GeomCLIP framework to enhance for multi-modal representation learning from molecular structures and biomedical text. During pre-training, we design two types of tasks, i.e., multimodal representation alignment and unimodal denoising pretraining, to align the 3D geometric encoder with textual information and, at the same time, preserve its original representation power. Experimental results show the effectiveness of GeomCLIP in various tasks such as molecular property prediction, zero-shot text-molecule retrieval, and 3D molecule captioning. Our code and collected dataset are available at \url{https://github.com/xiaocui3737/GeomCLIP}

📄 PDF Abstract BibTeX arXiv:2411.10821

Code (1)

xiaocui3737/geomclip 공식 구현 pytorch

Tasks

Denoising Molecular Property Prediction Molecule CaptioningProperty PredictionRepresentation Learning

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

ALIGN In the ALIGN method, visual and language representations are jointly trained from noisy image alt-text data. The image and text encoders are learned via contrastive loss…

Focus 설명 없음

3D Interaction Geometric Pre-training for Molecular Relational Learning

2024-12-04 · Namkyeong Lee, Yunhak Oh, Heewoong Noh, Gyoung S. Na 외

Molecular Relational Learning (MRL) is a rapidly growing field that focuses on understanding the interaction dynamics between molecules, which is crucial for applications ranging from catalyst engineering to drug discove…

Contrastive Learning Drug Discovery Relational Reasoning

SALSA: Semantically-Aware Latent Space Autoencoder

2023-10-04 · Kathryn E. Kirchoff, Travis Maxfield, Alexander Tropsha, Shawn M. Gomez

In deep learning for drug discovery, chemical data are often represented as simplified molecular-input line-entry system (SMILES) sequences which allow for straightforward implementation of natural language processing me…

Drug Discovery Graph Similarity

Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editing

2022-12-21 · Shengchao Liu, Weili Nie, Chengpeng Wang, Jiarui Lu 외

There is increasing adoption of artificial intelligence in drug discovery. However, existing studies use machine learning to mainly utilize the chemical structures of molecules but ignore the vast textual knowledge avail…

Contrastive Learning Drug Design Drug Discovery Retrieval+1

Geometry-Complete Diffusion for 3D Molecule Generation and Optimization

2023-02-08 · Alex Morehead, Jianlin Cheng

Denoising diffusion probabilistic models (DDPMs) have pioneered new state-of-the-art results in disciplines such as computer vision and computational biology for diverse tasks ranging from text-guided image generation to…

3D Molecule Generation Denoising Graph Generation Image Generation+5

Molecular Contrastive Learning of Representations via Graph Neural Networks

2021-02-19 · Yuyang Wang, Jianren Wang, Zhonglin Cao, Amir Barati Farimani

Molecular Machine Learning (ML) bears promise for efficient molecule property prediction and drug discovery. However, labeled molecule data can be expensive and time-consuming to acquire. Due to the limited labeled data,…

BIG-bench Machine Learning Contrastive Learning Drug Discovery Molecular Property Prediction+2