DSSLP: A Distributed Framework for Semi-supervised Link Prediction
Link prediction is widely used in a variety of industrial applications, such as merchant recommendation, fraudulent transaction detection, and so on. However, it's a great challenge to train and deploy a link prediction model on industrial-scale graphs with billions of nodes and edges. In this work, we present a scalable and distributed framework for semi-supervised link prediction problem (named DSSLP), which is able to handle industrial-scale graphs. Instead of training model on the whole graph, DSSLP is proposed to train on the \emph{$k$-hops neighborhood} of nodes in a mini-batch setting, which helps reduce the scale of the input graph and distribute the training procedure. In order to generate negative examples effectively, DSSLP contains a distributed batched runtime sampling module. It implements uniform and dynamic sampling approaches, and is able to adaptively construct positive and negative examples to guide the training process. Moreover, DSSLP proposes a model-split strategy to accelerate the speed of inference process of the link prediction task. Experimental results demonstrate that the effectiveness and efficiency of DSSLP in serval public datasets as well as real-world datasets of industrial-scale graphs.
Code (0)
등록된 구현이 없습니다.
Tasks
Link PredictionPredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Memetic Differential Evolution Methods for Semi-Supervised Clustering
In this paper, we propose an extension for semi-supervised Minimum Sum-of-Squares Clustering (MSSC) problems of MDEClust, a memetic framework based on the Differential Evolution paradigm for unsupervised clustering. In s…
ClusteringA Classification-Based Approach to Semi-Supervised Clustering with Pairwise Constraints
In this paper, we introduce a neural network framework for semi-supervised clustering (SSC) with pairwise (must-link or cannot-link) constraints. In contrast to existing approaches, we decompose SSC into two simpler clas…
Binary ClassificationClassificationClusteringGeneral ClassificationSemi-supervised Gender Classification with Joint Textual and Social Modeling
In gender classification, labeled data is often limited while unlabeled data is ample. This motivates semi-supervised learning for gender classification to improve the performance by exploring the knowledge in both label…
ClassificationGender ClassificationGeneral ClassificationSemi-Supervised Information-Maximization Clustering
Semi-supervised clustering aims to introduce prior knowledge in the decision process of a clustering algorithm. In this paper, we propose a novel semi-supervised clustering algorithm based on the information-maximization…
ClusteringColdExpand: Semi-Supervised Graph Learning in Cold Start
Most real-world graphs are dynamic and eventually face the cold start problem. A fundamental question is how the new cold nodes acquire initial information in order to be adapted in to the existing graph. Here we propose…
Graph LearningLink PredictionNode ClassificationPrediction