Graph Machine Learning

ICML 2020 collaboration graph As a preview to my future post (next week) about ICML 2020, I want to share a collaboration graph between different organizations. Final graph has 429 nodes (organizations) and 1206 edges (collaborations). Each edge has a weight: the number of papers the organizations collaborated with. As the final graph is too big to display nicely, you can also look at the subgraph between organizations that collaborated the most (at least 30 collaborations). I will release a colab notebook so that you can play with it.

Are Hyperbolic Representations in Graphs Created Equal? The second submission to GRL workshop was on hyperbolic embeddings for graphs. We first make a good introduction to the distances and dot products in k-Stereographic model (a Riemannian manifold with constant curvature) and fix the issue with taking gradients at zero curvature, by taking a Taylor series expansion around the origin. This allows seamless gradient descent optimization in non-Euclidean space. Then we make experiments on node and graph classification, link prediction, and graph embedding task (i.e. preserving distances in the latent space) and show that for link prediction and graph embedding there is an uplift in using hyperbolic manifolds, while for node and graph classification Euclidean models work better.

Gradient Boosting Meets Graph Neural Networks for Heterogeneous Data We have two short paper submissions this year to GRL workshop this year. One of them is about application of gradient boosting decision trees (GBDT) to graphs. We know that Xgboost, LightGBM, and CatBoost perform extremely well on tabular data and are preferred methods for competitions like Kaggle. But how do you generalize it to graph-structured data? A naïve approach is to train first GBDT on node features only, ignoring graph topology and then use predictions as additional features to your model. But that misses graph information, possibly leading to inaccurate predictions. Instead, we propose to train GBDT and GNN end-to-end such that each tree of GBDT approximates mistakes made by GNN in the forward passes. We call the model Boosted Graph Neural Network and show that it can lead to significant uplift in performance in node regression task, while being very efficient.

Fresh picks from ArXiv This week is rich on explainable GNN literature, as well as papers on compression of graphs, combinatorial optimization and recommendation. GNN • Learning to Solve Combinatorial Optimization Problems on Real-World Graphs in Linear Time • Are Graph Convolutional Networks Fully Exploiting Graph Structure? • Accurately Solving Physical Systems with Graph Learning •Single-Layer Graph Convolutional Networks For Recommendation •XAI for Graphs: Explaining Graph Neural Network Predictions by Identifying Relevant Walks •XGNN: Towards Model-Level Explanations of Graph Neural Networks •Universal Graph Compression: Stochastic Block Models •Convergence and Stability of Graph Convolutional Networks on Large Random Graphs •Fairness-Aware Explainable Recommendation over Knowledge Graphs Graph Theory • Hierarchical hyperbolicity of graph products • Tree-Projected Gradient Descent for Estimating Gradient-Sparse Parameters on Graphs • The Weisfeiler-Leman dimension of chordal bipartite graphs without bipartite claw Conferences • Multi-level Graph Convolutional Networks for Cross-platform Anchor Link Prediction KDD 20 Surveys • Generate FAIR Literature Surveys with Scholarly Knowledge Graphs •A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions

ICML 2020 arxiv links There is a nice website that gather all (available) links to papers at ICML. There are some interesting insights. First, it's interesting to see what is the oldest paper that was accepted to ICML this year. Apparently this paper was published on arxiv in Sep 2017, waiting for a little bit less than 3 years to get accepted. There are 8 papers from 2018. And the authors probably started working on these papers 6-9 months before the publication date. It's brutal. Another interesting observation is that the word graph appeared to be top-5 word among all words in titles, which show increased interest in graphs at ICML.

Graph Machine Learning research groups: Joan Bruna I do a series of posts on the groups in graph research, previous post is here. The seventh is Joan Bruna. He was one of the authors of the survey Geometric Deep Learning: going beyond Euclidean Data and now has increasingly more papers on the theoretical explanations of GNN. Joan Bruna (~1981) - Affiliation: New York University - Education: Ph.D. at Ecole Polytechnique, France in 2013 (supervised by Stephane Mallat); - h-index: 27; - Awards: NSF career award, Sloan fellowship, ICMLA best paper; - Interests: GNN theory, equivariant networks

Online Seminar on Mathematical Foundations of Data Science Virtual weekly seminars with excellent list of speakers, open to the public, on mathematics and statistics in ML.

Of course, there are many big names from graph machine learning for ICML 20 such as Jure Leskovec, Tom Jaakkola, Le Song, Peter Battaglia and others.

ICML 2020 stats ICML is the top conference in ML. Dates: July 12-18 Where: Online • 4990 submissions (vs 3424 in 2019) • 1088 accepted (vs 774 in 2019) • 21.8% acceptance rate (vs 22.6% in 2019) • 53 graph papers (5% of total)

Fresh picks from ArXiv This week has GNN variants for various types of graphs, NP-completeness of MaxCut problem, and a survey on graph data management 🖥 GNN • Understanding the Message Passing in Graph Neural Networks via Power Iteration • Interpretable and Efficient Heterogeneous Graph Convolutional Network • Graph Neural Network for Hamiltonian-Based Material Property Prediction • Non-IID Graph Neural Networks • Hierarchical Fashion Graph Network for Personalized Outfit Recommendation • Non-Local Graph Neural Networks Graph Theory • Complexity of Maximum Cut on Interval Graphs • Planar Graphs that Need Four Pages Surveys • Benchmarking Graph Data Management and Processing Systems: A Survey