The Effects of Randomness on the Stability of Node Embeddings
We systematically evaluate the (in-)stability of state-of-the-art node embedding algorithms due to randomness, i.e., the random variation of their outcomes given identical algorithms and graphs. We apply five node embeddings algorithms---HOPE, LINE, node2vec, SDNE, and GraphSAGE---to synthetic and empirical graphs and assess their stability under randomness with respect to (i) the geometry of embedding spaces as well as (ii) their performance in downstream tasks. We find significant instabilities in the geometry of embedding spaces independent of the centrality of a node. In the evaluation of downstream tasks, we find that the accuracy of node classification seems to be unaffected by random seeding while the actual classification of nodes can vary significantly. This suggests that instability effects need to be taken into account when working with node embeddings. Our work is relevant for researchers and engineers interested in the effectiveness, reliability, and reproducibility of node embedding approaches.
Code (2)
Tasks
General ClassificationNode ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
The Impact of Dimensionality on the Stability of Node Embeddings
Previous work has shown that node embedding methods can produce different representations and downstream predictions across repeated training runs, even when trained on the same data with identical hyperparameters. Howev…
A Survey on Stability of Learning with Limited Labelled Data and its Sensitivity to the Effects of Randomness
Learning with limited labelled data, such as prompting, in-context learning, fine-tuning, meta-learning or few-shot learning, aims to effectively train a model using only a small amount of labelled samples. However, thes…
Few-Shot LearningIn-Context LearningMeta-LearningSensitivity+2On the Surprising Behaviour of node2vec
Graph embedding techniques are a staple of modern graph learning research. When using embeddings for downstream tasks such as classification, information about their stability and robustness, i.e., their susceptibility t…
Graph EmbeddingGraph LearningOn Sensitivity of Learning with Limited Labelled Data to the Effects of Randomness: Impact of Interactions and Systematic Choices
While learning with limited labelled data can improve performance when the labels are lacking, it is also sensitive to the effects of uncontrolled randomness introduced by so-called randomness factors (e.g., varying orde…
In-Context LearningMeta-LearningSensitivitytext-classification+1USER: Unsupervised Structural Entropy-based Robust Graph Neural Network
Unsupervised/self-supervised graph neural networks (GNN) are vulnerable to inherent randomness in the input graph data which greatly affects the performance of the model in downstream tasks. In this paper, we alleviate t…
Graph Neural NetworkLink PredictionNode Clustering