Geometric Transformer for End-to-End Molecule Properties Prediction
Transformers have become methods of choice in many applications thanks to their ability to represent complex interactions between elements. However, extending the Transformer architecture to non-sequential data such as molecules and enabling its training on small datasets remains a challenge. In this work, we introduce a Transformer-based architecture for molecule property prediction, which is able to capture the geometry of the molecule. We modify the classical positional encoder by an initial encoding of the molecule geometry, as well as a learned gated self-attention mechanism. We further suggest an augmentation scheme for molecular data capable of avoiding the overfitting induced by the overparameterized architecture. The proposed framework outperforms the state-of-the-art methods while being based on pure machine learning solely, i.e. the method does not incorporate domain knowledge from quantum chemistry and does not use extended geometric inputs besides the pairwise atomic distances.
Code (1)
Tasks
Property PredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Edge Direction-invariant Graph Neural Networks for Molecular Dipole Moments Prediction
The dipole moment is a physical quantity indicating the polarity of a molecule and is determined by reflecting the electrical properties of constituent atoms and the geometric properties of the molecule. Most embeddings …
Graph Neural NetworkMolecule Design by Latent Prompt Transformer
This work explores the challenging problem of molecule design by framing it as a conditional generative modeling task, where target biological properties or desired chemical constraints serve as conditioning variables. W…
Property PredictionTransformers for molecular property prediction: Domain adaptation efficiently improves performance
Most of the current transformer-based chemical language models are pre-trained on millions to billions of molecules. However, the improvement from such scaling in dataset size is not confidently linked to improved molecu…
Domain AdaptationMolecular Property PredictionProperty PredictionGeomGCL: Geometric Graph Contrastive Learning for Molecular Property Prediction
Recently many efforts have been devoted to applying graph neural networks (GNNs) to molecular property prediction which is a fundamental task for computational drug and material discovery. One of major obstacles to hinde…
Contrastive LearningData AugmentationMolecular Property Predictionmolecular representation+2Geometric Transformer with Interatomic Positional Encoding
The widespread adoption of Transformer architectures in various data modalities has opened new avenues for the applications in molecular modeling. Nevertheless, it remains elusive that whether the Transformer-based arch…