Cardinality-Preserving Attention Channels for Graph Transformers in Molecular Property Prediction
Molecular property prediction is crucial for drug discovery when labeled data are scarce. This work presents CardinalGraphFormer, a graph transformer augmented with a query-conditioned cardinality-preserving attention (CPA) channel that retains dynamic support-size signals complementary to static centrality embeddings. The approach combines structured sparse attention with Graphormer-inspired biases (shortest-path distance, centrality, direct-bond features) and unified dual-objective self-supervised pretraining (masked reconstruction and contrastive alignment of augmented views). Evaluation on 11 public benchmarks spanning MoleculeNet, OGB, and TDC ADMET demonstrates consistent improvements over protocol-matched baselines under matched pretraining, optimization, and hyperparameter tuning. Rigorous ablations confirm CPA's contributions and rule out simple size shortcuts. Code and reproducibility artifacts are provided.
Code (0)
등록된 구현이 없습니다.
Tasks
Molecular Property PredictionDrug DiscoverySimilar Papers 제목 키워드 기반
CardiGraphormer: Unveiling the Power of Self-Supervised Learning in Revolutionizing Drug Discovery
In the expansive realm of drug discovery, with approximately 15,000 known drugs and only around 4,200 approved, the combinatorial nature of the chemical space presents a formidable challenge. While Artificial Intelligenc…
Drug DiscoverySelf-Supervised LearningPrivacy-Preserving Record Linkage for Cardinality Counting
Several applications require counting the number of distinct items in the data, which is known as the cardinality counting problem. Example applications include health applications such as rare disease patients counting …
ClusteringMarketingPrivacy PreservingEquiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs
Despite their widespread success in various domains, Transformer networks have yet to perform well across datasets in the domain of 3D atomistic graphs such as molecules even when 3D-related inductive biases like transla…
Graph AttentionGraph Neural NetworkGraph Property PredictionInitial Structure to Relaxed Energy (IS2RE), Direct+1Improving Attention Mechanism in Graph Neural Networks via Cardinality Preservation
Graph Neural Networks (GNNs) are powerful to learn the representation of graph-structured data. Most of the GNNs use the message-passing scheme, where the embedding of a node is iteratively updated by aggregating the inf…
Graph ClassificationGraph Representation LearningNode ClassificationTransformers through the lens of support-preserving maps between measures
Transformers are deep architectures that define ``in-context maps'' which enable predicting new tokens based on a given set of tokens (such as a prompt in NLP applications or a set of patches for a vision transformer). I…