Graph Machine Learning

Modeling Intelligence via Graph Neural Networks: slides The slides of the thesis by Keyulu Xu: Modeling Intelligence via Graph Neural Networks. Keyulu is one of the authors of GIN and other notable works in GML.

Stanford Graph Learning Workshop A great online workshop will be organized by Stanford, on Thursday, Sept 16 2021, 08:00 - 17:00 Pacific Time. It includes talks from Jure Leskovec, Matthias Fey, Weihua Hu, Jiaxuan You, as well as a series of talks on applications of GNNs, and two industry panels.

Fresh picks from ArXiv This week on ArXiv: GNN link with causal models, augmenting data, and using knowledge graphs with BERT 🧸 If I forgot to mention your paper, please shoot me a message and I will update the post. GNNs * Relating Graph Neural Networks to Structural Causal Models with Petar Veličković, Kristian Kersting * A Study of Joint Graph Inference and Forecasting with Stephan Günnemann * Is Heterophily A Real Nightmare For Graph Neural Networks To Do Node Classification? * Local Augmentation for Graph Neural Networks * Mixture-of-Partitions: Infusing Large Biomedical Knowledge Graphs into BERT EMNLP 2021 Math * There does not exist a strongly regular graph with parameters (1911,270,105,27) * On the Fundamental Limits of Matrix Completion: Leveraging Hierarchical Similarity Graphs

PyG 2.0 (PyTorch Geometric 2.0) Release One of the most prominent libraries in the world of GNNs and Geometric DL got a major update (and a small re-branding to a shorter "PyG")! Now with a website and Slack channel. In addition to a constantly growing number of supported GNN architectures, the 2.0 version features: 1. Heterogeneous graph support with models, mini-batching, sampling, and a one-line conversion of homogeneous models to heterogeneous. 2. GraphGym - a whole platform for designing and experimenting with GNN architectures where you can fine-tune the nitty-gritty details of your model and find the best hyperparams. Based on the NeurIPS'20 paper 3. Pre-defined models - before, you'd usually build a GNN model from a collection of layers by yourself (trying to not forget to put that non-linearity after the GCN layer). Now, the library includes 25 well-known models! 4. Half-precision support and other smaller improvements to make your GNN journey easier.

Organizational Update I've been running this channel alone for almost two years but it's been more challenging recently to keep the previous pace. To help me, Michael Galkin generously accepted to be one of the admins of this channel, who has been already involved in several posts here. Michael Galkin is a postdoc at Mila & McGill and you can know him by the amazing digests of knowledge graphs papers, contributions to the open-source projects, and strong research works. Please, welcome Michael and subscribe to his twitter. P.S. Also I will use this opportunity to remind that if you have something to share with a graph community, do not hesitate to contact us.

Review: Deep Learning on Sets A new blog post by Fabian Fuchs and others about recent approaches of applying deep learning on sets. It digests several paradigms such as permuting & averaging, sorting, approximating invariance, and learning on graphs as a way to overcome permutation invariance of machine learning algorithms.

Graph ML in Industry Workshop When I wrote top applications of GNNs at the beginning of this year, I had a feeling that graph ML community is mature enough to start being used in industrial companies. Nine months ahead we decided to gather researchers, engineers, and industry professionals to talk about applications of graphs in the companies. Please, join us on 23rd Sept, 17-00 Paris time (free, online, ~3 hours) by registering at the link.

Researcher Positions at Dimitri's Ognibene's Lab Two positions for post doc/researchers are available at Milano Bicocca University under Dimitri's Ognibene supervision. 2 years contract, based in Milan (possibility to remote working). For application contact: Dimitri Ognibene [email protected]. Description is below: Do social media harm teenagers and our society? Can we make them safer? We will use the state of the art in graph neural networks, reinforcement learning, nlp, cv, and machine learning in general to improve our understanding of social media dynamics, and help our society by supporting and teaching young people tackle hate speech and fake news in social media.

The Learning on Graphs and Geometry Reading Group A reading group organized by Hannes Stärk with supervision from Pietro Liò at Cambridge. Includes really interesting fresh papers on graphs. Every Tuesday at 5pm CEST.

Fresh picks from ArXiv This week on ArXiv: improving robustness by resampling a graph, learning better scenes, and new homophily definitions 🐤 If I forgot to mention your paper, please shoot me a message and I will update the post. GNNs * Training Graph Neural Networks by Graphon Estimation * Learning to Generate Scene Graph from Natural Language Supervision ICCV 2021 * Pointspectrum: Equivariance Meets Laplacian Filtering for Graph Representation Learning * Sparsifying the Update Step in Graph Neural Networks * Adaptive Label Smoothing To Regularize Large-Scale Graph Training Math * How likely is a random graph shift-enabled? * The Popularity-Homophily Index: A new way to measure Homophily in Directed Graphs

Monday Theory: Structural vs Positional Node Representations In the new slide deck, Bruno Ribeiro (Purdue University) uncovers the nature of two commonly used mechanisms for building node representations. Structural representations are permutation insensitive (like GNNs) whereas positional representations are permutation sensitive (like SVD vectors). Hence, all GRL approaches can be broadly classified into those two families. Takeaway messages: Message 1: Positional representations of k nodes are to most expressive k-node structural representations as samples of a distribution are to sufficient statistics of the distribution. This is based on the results published in the ICLR'20 paper Message 2: As soon as you introduce some sort of node IDs you break equivariance but at the same time can predict properties of any subset of nodes (better link prediction). You’d better aggregate over multiple samples though (from the stats analogy). If you stick to equivariance, you can predict node or graph-level properties but nothing in-between.

GNN Tutorial & Graph Convolution Intuition @ Distill Distill.pub is a great new resource aimed at re-defining a way we publish papers. Publications on Distill have rich visualizations and hands-on examples that you can tweak right in a browser. Unfortunately, Distill goes on a hiatus. But, as the last bow, the authors prepared two very cool articles breaking down message passing and graph convolutions: 1. A Gentle Introduction to Graph Neural Networks 2. Understanding Convolutions on Graphs Something you definitely do not want to miss in September!