Using pretrained graph neural networks with token mixers as geometric featurizers for conformational dynamics
Identifying informative low-dimensional features that characterize dynamics in molecular simulations remains a challenge, often requiring extensive manual tuning and system-specific knowledge. Here, we introduce geom2vec, in which pretrained graph neural networks (GNNs) are used as universal geometric featurizers. By pretraining equivariant GNNs on a large dataset of molecular conformations with a self-supervised denoising objective, we obtain transferable structural representations that are useful for learning conformational dynamics without further fine-tuning. We show how the learned GNN representations can capture interpretable relationships between structural units (tokens) by combining them with expressive token mixers. Importantly, decoupling training the GNNs from training for downstream tasks enables analysis of larger molecular graphs (such as small proteins at all-atom resolution) with limited computational resources. In these ways, geom2vec eliminates the need for manual feature selection and increases the robustness of simulation analyses.
Code (2)
Tasks
Denoisingfeature selectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Shaken or Stirred? An Analysis of MetaFormer's Token Mixing for Medical Imaging
The generalization of the Transformer architecture via MetaFormer has reshaped our understanding of its success in computer vision. By replacing self-attention with simpler token mixers, MetaFormer provides strong baseli…
Semantic SegmentationImage ClassificationStructuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds
What is the geometry of a visual percept? The most widely used protocols for decomposing neural network representations into interpretable parts treat concepts as isolated directions, yet recent work shows that concepts …
Image GenerationGroupedMixer: An Entropy Model with Group-wise Token-Mixers for Learned Image Compression
Transformer-based entropy models have gained prominence in recent years due to their superior ability to capture long-range dependencies in probability distribution estimation compared to convolution-based methods. Howev…
Image CompressionMetaFormer Baselines for Vision
MetaFormer, the abstracted architecture of Transformer, has been found to play a significant role in achieving competitive performance. In this paper, we further explore the capacity of MetaFormer, again, without focusin…
Domain GeneralizationImage ClassificationAdaptive Frequency Filters As Efficient Global Token Mixers
Recent vision transformers, large-kernel CNNs and MLPs have attained remarkable successes in broad vision tasks thanks to their effective information fusion in the global scope. However, their efficient deployments, espe…