Coordinating Cross-modal Distillation for Molecular Property Prediction
In recent years, molecular graph representation learning (GRL) has drawn much more attention in molecular property prediction (MPP) problems. The existing graph methods have demonstrated that 3D geometric information is significant for better performance in MPP. However, accurate 3D structures are often costly and time-consuming to obtain, limiting the large-scale application of GRL. It is an intuitive solution to train with 3D to 2D knowledge distillation and predict with only 2D inputs. But some challenging problems remain open for 3D to 2D distillation. One is that the 3D view is quite distinct from the 2D view, and the other is that the gradient magnitudes of atoms in distillation are discrepant and unstable due to the variable molecular size. To address these challenging problems, we exclusively propose a distillation framework that contains global molecular distillation and local atom distillation. We also provide a theoretical insight to justify how to coordinate atom and molecular information, which tackles the drawback of variable molecular size for atom information distillation. Experimental results on two popular molecular datasets demonstrate that our proposed model achieves superior performance over other methods. Specifically, on the largest MPP dataset PCQM4Mv2 served as an "ImageNet Large Scale Visual Recognition Challenge" in the field of graph ML, the proposed method achieved a 6.9% improvement compared with the best works. And we obtained fourth place with the MAE of 0.0734 on the test-challenge set for OGB-LSC 2022 Graph Regression Task. We will release the code soon.
Code (0)
등록된 구현이 없습니다.
Tasks
Graph RegressionGraph Representation LearningKnowledge DistillationMolecular Property PredictionObject RecognitionPredictionProperty PredictionRepresentation LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MolKD: Distilling Cross-Modal Knowledge in Chemical Reactions for Molecular Property Prediction
How to effectively represent molecules is a long-standing challenge for molecular property prediction and drug discovery. This paper studies this problem and proposes to incorporate chemical domain knowledge, specificall…
Drug DiscoveryMolecular Property PredictionPredictionProperty Prediction3D Denoisers are Good 2D Teachers: Molecular Pretraining via Denoising and Cross-Modal Distillation
Pretraining molecular representations from large unlabeled data is essential for molecular property prediction due to the high cost of obtaining ground-truth labels. While there exist various 2D graph-based molecular pre…
DenoisingKnowledge DistillationMolecular Property Predictionmolecular representation+2LLM and GNN are Complementary: Distilling LLM for Multimodal Graph Learning
Recent progress in Graph Neural Networks (GNNs) has greatly enhanced the ability to model complex molecular structures for predicting properties. Nevertheless, molecular data encompasses more than just graph structures, …
Graph LearningLanguage ModelingLanguage ModellingLarge Language ModelMulti-Modal Representation Learning for Molecular Property Prediction: Sequence, Graph, Geometry
Molecular property prediction refers to the task of labeling molecules with some biochemical properties, playing a pivotal role in the drug discovery and design process. Recently, with the advancement of machine learning…
Data AugmentationDrug DiscoveryMolecular Property Predictionmolecular representation+3MOLAR: Learning Multimodal Molecular Representations from Noisy Labels
Motivation: Noisy labels are a common challenge in molecular property prediction because molecular annotations are often obtained from assays, curated databases, or weak annotation pipelines rather than directly observed…
Molecular Property PredictionRepresentation Learning