Graph Machine Learning

Drawing neural networks in LaTeX There is a repo of good examples by Petar Veličković of how you can draw Tikz images in LaTeX. Here is an example of 1-layer GNN by Matthias Fey.

The Quantum Graph Recurrent Neural Network This demonstration by pennylane investigates quantum graph recurrent neural networks (QGRNN), which are the quantum analogue of a classical graph recurrent neural network, and a subclass of the more general quantum graph neural network ansatz. Both the QGNN and QGRNN were introduced in this paper (2019) by Google X.

Israeli Geometric Deep Learning Workshop Many cool presentations at the recent DGL workshop, including Yaron Lipman, Gal Chechik, Gal Chechik, and many other experienced people in this field. The video is on YouTube.

Fresh picks from ArXiv This week discusses ways to compute a distance between graphs, fast triangle counting, and a new i.i.d. test 🧪 GNN • Adversarial Directed Graph Embedding with Philip S. Yu • CoCoS: Fast and Accurate Distributed Triangle Counting in Graph Streams with Christos Faloutsos • Network comparison and the within-ensemble graph distance Math • Using Expander Graphs to test whether samples are i.i.d Surveys • Graph Signal Processing for Geometric Data and Beyond: Theory and Applications • Graph Neural Networks: Architectures, Stability and Transferability • Random Walks: A Review of Algorithms and Applications • Big Networks: A Survey

MLSS Indo 2020: Equivariance MLSS-Indonesia is part of MLSS series, which is one of the top summer schools on ML. Here is a lecture and slides on equivariance and group theory and how it's used for convolutions.

Simple scalable graph neural networks Michael Bronstein continues a marathon of great blog posts on GML. In a new post he describes their recent work on scaling GNNs to large network. There is a good introduction to sampling-based methods (e.g. SAGE, GraphSAINT, ClusterGCN), which sample a subgraph of a large graph and then train GNN only on a subgraph. Then, he describes that it can be beneficial just precompute r-hop matrices, A^r X, and use MLP on these features. This way, you use topology of your graph and you apply mini-batch training with MLP. What's cool is that the algorithm is already available in pytorch-geometric as a transform, which is implemented via sparseTensor matrix multiplication.

Issue #1: Introduction, PAC Isometry, Over-Smoothing, and Evolution of the Field Finally the first issue of a newsletter is out and I hope there will many more in the future. The most difficult of this is to find good stories for the email: it's somewhat different from posting on telegram and twitter, as you need to have more insights in a single story. So if you find something that could be relevant to the community, definitely send me a message.

Probabilistic Learning on Graphs via Contextual Architectures This is a guest post by Federico Errica ([email protected]) about their new JMLR work called “Probabilistic Learning on Graphs via Contextual Architectures”. Intro/TL;DR: We propose a probabilistic methodology for representation learning on graph-structured data, in which a stack of Bayesian networks learns different distributions of a vertex’s neighbourhood. The main characteristics of our approach are (i) unsupervised, as it models the generation of node attributes; (ii) layer-wise training: (iii) incremental construction policy; (iv) maximum likelihood estimation with Expectation-Maximization. The model, called Contextual Graph Markov Model (CGMM), can be regarded as a probabilistic version of Deep Graph Networks (DGNs). Each layer of the model implements a probabilistic version of neighbourhood aggregation. The hidden representation of each node is modelled as a categorical distribution. When aggregating neighbours, the incoming messages are the *frozen* posterior probabilities computed when training the previous layers. When discrete edge types are available, we can weight the contribution of nodes in different ways using the Switching Parent approximation. Moreover, each neighbour aggregation can be conditioned on an arbitrary subset of the previous layers. By design, this incremental construction policy avoids the exploding/vanishing gradient effect. As a result, each layer exploits different sets of statistics when trying to maximize the likelihood of the nodes in each graph. We test the model on node and graph classification tasks. First, we generate unsupervised node/graph representations; then, we apply a standard ML classifier to output the right class. In turn, this leads to a critical analysis of some benchmarks used in the literature. Finally, we show that the performances of the model increase as we add more layers (up to 20). Paper: http://www.jmlr.org/papers/v21/19-470.html Code: https://github.com/diningphil/CGMM Related reads: (i) https://doi.org/10.1016/j.neunet.2020.06.006 (ii) http://proceedings.mlr.press/v80/bacciu18a.html

Number of papers in GML There are 339 new GML papers in CS section of ArXiv in July 2020.

Videos for ICML 2020 workshops and tutorials Available at slideslive. Two related to GML are: • GRL+ workshop • Bridge Between Perception and Reasoning: Graph Neural Networks & Beyond workshop

Fresh picks from ArXiv This week studies architecture search and low latency inference in GNN as well as review on graph signal processing 📶 GNN • Neural Architecture Search in Graph Neural Networks • FC-GAGA: Fully Connected Gated Graph Architecture for Spatio-Temporal Traffic Forecasting • Pooling Regularized Graph Neural Network for fMRI Biomarker Analysis • GRIP: A Graph Neural Network Accelerator Architecture with Christopher Re • PyKEEN 1.0: A Python Library for Training and Evaluating Knowledge Graph Embeddings Math • A polynomial-time algorithm to determine (almost) Hamiltonicity of dense regular graphs Surveys • Graph signal processing for machine learning: A review and new perspectives with Michael Bronstein