paper-with-me

홈 › Papers

Analysis and Approximate Inference of Large Random Kronecker Graphs

2023-06-14 · Zhenyu Liao, Yuanqian Xia, Chengmei Niu, Yong Xiao

Random graph models are playing an increasingly important role in various fields ranging from social networks, telecommunication systems, to physiologic and biological networks. Within this landscape, the random Kronecker graph model, emerges as a prominent framework for scrutinizing intricate real-world networks. In this paper, we investigate large random Kronecker graphs, i.e., the number of graph vertices $N$ is large. Built upon recent advances in random matrix theory (RMT) and high-dimensional statistics, we prove that the adjacency of a large random Kronecker graph can be decomposed, in a spectral norm sense, into two parts: a small-rank (of rank $O(\log N)$) signal matrix that is linear in the graph parameters and a zero-mean random noise matrix. Based on this result, we propose a ``denoise-and-solve'' approach to infer the key graph parameters, with significantly reduced computational complexity. Experiments on both graph inference and classification are presented to evaluate the our proposed method. In both tasks, the proposed approach yields comparable or advantageous performance, than widely-used graph inference (e.g., KronFit) and graph neural net baselines, at a time cost that scales linearly as the graph size $N$.

📄 PDF Abstract BibTeX arXiv:2306.08489

Code (1)

yqian108/inference-of-kronecker-graph 공식 구현 pytorch

Tasks

Graph ClassificationRecommendation Systems

Similar Papers 제목 키워드 기반

Approximating Real-Time Recurrent Learning with Random Kronecker Factors

2018-05-28 · NeurIPS 2018 12 · Asier Mujika, Florian Meier, Angelika Steger

Despite all the impressive advances of recurrent neural networks, sequential data is still in need of better modelling. Truncated backpropagation through time (TBPTT), the learning algorithm most widely used in practice,…

Memorization

An iterative K-FAC algorithm for Deep Learning

2021-01-01 · Yingshi Chen

Kronecker-factored Approximate Curvature (K-FAC) method is a high efficiency second order optimizer for the deep learning. Its training time is less than SGD(or other first-order method) with same accuracy in many large-…

Deep Learning

Thoughts on Massively Scalable Gaussian Processes

2015-11-05 · Andrew Gordon Wilson, Christoph Dann, Hannes Nickisch

We introduce a framework and early results for massively scalable Gaussian processes (MSGP), significantly extending the KISS-GP approach of Wilson and Nickisch (2015). The MSGP framework enables the use of Gaussian proc…

Gaussian Processes

Stochastic Neural Network with Kronecker Flow

2019-06-10 · Chin-wei Huang, Ahmed Touati, Pascal Vincent, Gintare Karolina Dziugaite 외

Recent advances in variational inference enable the modelling of highly structured joint distributions, but are limited in their capacity to scale to the high-dimensional setting of stochastic neural networks. This limit…

Multi-Armed BanditsThompson SamplingVariational Inference

A Kronecker-factored approximate Fisher matrix for convolution layers

2016-02-03 · Roger Grosse, James Martens

Second-order optimization methods such as natural gradient descent have the potential to speed up training of neural networks by correcting for the curvature of the loss function. Unfortunately, the exact natural gradien…

Stochastic Optimization