Curvature-informed multi-task learning for graph networks
Properties of interest for crystals and molecules, such as band gap, elasticity, and solubility, are generally related to each other: they are governed by the same underlying laws of physics. However, when state-of-the-art graph neural networks attempt to predict multiple properties simultaneously (the multi-task learning (MTL) setting), they frequently underperform a suite of single property predictors. This suggests graph networks may not be fully leveraging these underlying similarities. Here we investigate a potential explanation for this phenomenon: the curvature of each property's loss surface significantly varies, leading to inefficient learning. This difference in curvature can be assessed by looking at spectral properties of the Hessians of each property's loss function, which is done in a matrix-free manner via randomized numerical linear algebra. We evaluate our hypothesis on two benchmark datasets (Materials Project (MP) and QM8) and consider how these findings can inform the training of novel multi-task learning models.
Code (0)
등록된 구현이 없습니다.
Tasks
Band GapMulti-Task LearningSimilar Papers 제목 키워드 기반
Curvature-Informed Potential Energy Surface for Protein-Ligand Binding Affinity Prediction
Accurate prediction of protein-ligand binding affinity is essential for structure-based drug discovery. Recent geometric deep learning methods have achieved promising performance by representing protein-ligand complexes …
Graph Neural NetworkDrug DiscoveryR-ODE: Ricci Curvature Tells When You Will be Informed
Information diffusion prediction is fundamental to understand the structure and organization of the online social networks, and plays a crucial role to blocking rumor spread, influence maximization, political propaganda,…
BlockingGraph Neural NetworkProbing optimisation in physics-informed neural networks
A novel comparison is presented of the effect of optimiser choice on the accuracy of physics-informed neural networks (PINNs). To give insight into why some optimisers are better, a new approach is proposed that tracks t…
ACE-HGNN: Adaptive Curvature Exploration Hyperbolic Graph Neural Network
Graph Neural Networks (GNNs) have been widely studied in various graph data mining tasks. Most existingGNNs embed graph data into Euclidean space and thus are less effective to capture the ubiquitous hierarchical structu…
Graph LearningGraph Neural NetworkMulti-agent Reinforcement LearningRepresentation LearningMixed-Curvature Multi-Relational Graph Neural Network for Knowledge Graph Completion
Knowledge graphs (KGs) have gradually become valuable assets for many AI applications. In a KG, a node denotes an entity, and an edge (or link) denotes a relationship between the entities represented by the nodes. Knowle…
Graph Neural NetworkKnowledge Graph CompletionKnowledge Graph EmbeddingsKnowledge Graphs