paper-with-me

Papers

Using pretrained graph neural networks with token mixers as geometric featurizers for conformational dynamics

2024-09-30 · Zihan Pengmei, Chatipat Lorpaiboon, Spencer C. Guo, Jonathan Weare, Aaron R. Dinner

Identifying informative low-dimensional features that characterize dynamics in molecular simulations remains a challenge, often requiring extensive manual tuning and system-specific knowledge. Here, we introduce geom2vec, in which pretrained graph neural networks (GNNs) are used as universal geometric featurizers. By pretraining equivariant GNNs on a large dataset of molecular conformations with a self-supervised denoising objective, we obtain transferable structural representations that are useful for learning conformational dynamics without further fine-tuning. We show how the learned GNN representations can capture interpretable relationships between structural units (tokens) by combining them with expressive token mixers. Importantly, decoupling training the GNNs from training for downstream tasks enables analysis of larger molecular graphs (such as small proteins at all-atom resolution) with limited computational resources. In these ways, geom2vec eliminates the need for manual feature selection and increases the robustness of simulation analyses.

📄 PDF Abstract BibTeX arXiv:2409.19838

Code (2)

dinner-group/geom2vec 공식 구현 pytorch
zpengmei/geom2vec 공식 구현 pytorch

Tasks

Denoisingfeature selection

Methods 이 논문이 사용한 방법론

Feature Selection Feature selection, also known as variable selection, attribute selection or variable subset selection, is the process of selecting a subset of relevant features (variables,…

Similar Papers 제목 키워드 기반

Shaken or Stirred? An Analysis of MetaFormer's Token Mixing for Medical Imaging

2025-10-07 · Ron Keuth, Paul Kaftan, Mattias P. Heinrich arxiv

The generalization of the Transformer architecture via MetaFormer has reshaped our understanding of its success in computer vision. By replacing self-attention with simpler token mixers, MetaFormer provides strong baseli…

Semantic SegmentationImage Classification

Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds

2026-06-23 · Thomas Fel, Matthew Kowal, Mozes Jacobs, Dron Hazra 외 arxiv

What is the geometry of a visual percept? The most widely used protocols for decomposing neural network representations into interpretable parts treat concepts as isolated directions, yet recent work shows that concepts …

Image Generation

GroupedMixer: An Entropy Model with Group-wise Token-Mixers for Learned Image Compression

2024-05-02 · Daxin Li, Yuanchao Bai, Kai Wang, Junjun Jiang 외

Transformer-based entropy models have gained prominence in recent years due to their superior ability to capture long-range dependencies in probability distribution estimation compared to convolution-based methods. Howev…

Image Compression

MetaFormer Baselines for Vision

2022-10-24 · Weihao Yu, Chenyang Si, Pan Zhou, Mi Luo 외

MetaFormer, the abstracted architecture of Transformer, has been found to play a significant role in achieving competitive performance. In this paper, we further explore the capacity of MetaFormer, again, without focusin…

Domain GeneralizationImage Classification

Adaptive Frequency Filters As Efficient Global Token Mixers

2023-07-26 · ICCV 2023 1 · Zhipeng Huang, Zhizheng Zhang, Cuiling Lan, Zheng-Jun Zha 외

Recent vision transformers, large-kernel CNNs and MLPs have attained remarkable successes in broad vision tasks thanks to their effective information fusion in the global scope. However, their efficient deployments, espe…