Graph Machine Learning

A Tale of Three Implicit Planners and the XLVIN agent A video presentation by Petar Veličković about implicit planners, which could be seen as a middle-ground between model-based and model-free approaches for RL planning problems. The talk covers three popular implicit planners: VIN, ATreeC and XLVIN. All three focus on the recently popularised idea of algorithmically aligning to a planning algorithm, but with different realisations.

Large-scale graph machine learning challenge (OGB-LSC) at KDD Cup 2021 OGB-LSC is a collection of three graph datasets—PCQM4M-LSC, WikiKG90M-LSC, and MAG240M-LSC—that are orders of magnitude larger than existing ones. The three datasets correspond to link prediction, graph regression, and node classification tasks, respectively. The goal of OGB-LSC is to empower the community to discover innovative solutions for large-scale graph ML. The competition will be from March 15th, 2021 until June 8th, 2021 and the winners will be notified by mid-June 2021. The winners will be honored at the KDD 2021 opening ceremony and will present their solutions at the KDD Cup workshop during the conference. The graphs are indeed big, with the largest size 168 GB, and it's interesting what approaches can be used to solve these problems.

Fresh picks from ArXiv This week on ArXiv: comparison of using feature vs edge information, SOTA for heterogeneous graphs, surveys on efficient training of GNNs 🏃‍ If I forgot to mention your paper, please shoot me a message and I will update the post. GNNs * Spatio-temporal Modeling for Large-scale Vehicular Networks Using Graph Convolutional Networks * R-GSN: The Relation-based Graph Similar Network for Heterogeneous Graph * On the Equivalence Between Temporal and Static Graph Representations for Observational Predictions with Bruno Ribeiro * Deep Graph Matching under Quadratic Constraint * Should Graph Neural Networks Use Features, Edges, Or Both? * Deep graph convolution neural network with non-negative matrix factorization for community discovery * Graph Neural Networks Inspired by Classical Iterative Algorithms * Size-Invariant Graph Representations for Graph Classification Extrapolations Survey * Sampling methods for efficient training of graph convolutional networks: A survey * Graph Metrics for Internet Robustness -- A Survey * A Taxonomy for Classification and Comparison of Dataflows for GNN Accelerators

PyTorch Geometric Temporal PyG-Temporal is an extension of PyG for temporal graphs. It now includes more than 10 GNN models and several datasets. With world being dynamic I see more and more applications when standard GNN wouldn't work and one needs to resort to dynamic GNNs.

Graph Transformer: A Generalization of Transformers to Graphs A blog post by Vijay Prakash Dwivedi that discusses their paper A Generalization of Transformer Networks to Graphs with Xavier Bresson at 2021 AAAI Workshop (DLG-AAAI’21). It looks like a generalization of GAT network with batch norm and positional encodings. It still though aggregates via local neighborhoods. My feeling after studying heterophily is that we will see more works that go beyond local neighborhoods and maybe will define neighborhoods not as something that is given by the graph topology but as something we have to learn. For example, we can define attention from each node to all other nodes in the graph and treat the distances in the graph as additional features. It could be difficult to scale so sampling methods should be employed I guess, but it seems allowing the network to decide which nodes are important for aggregation could be a better way to go.

A Complete Beginner's Guide to G-Invariant Neural Networks A tutorial by S. Chandra Mouli and Bruno Ribeiro about G-invariant neural networks, eigenvectors, invariant subspaces, transformation groups, Reynolds operator, and more. Soon, there should be more tutorials on the topic of invariance and linear algebra.

If We Draw Graphs Like This, We Can Change Computers Forever The title is catchy, but the article is "only" about improvement for dynamic planarity testing problem. Planarity testing is well-studied problem for testing if a graph can be drawn without crossing edges and O(n) algorithms are known. This article on the other hand studies the case when the edges may be added and removed and the question is how to redraw the graph so that it becomes planar. The results were published at STOC'20.

Different styles of communication 😊

Deep Learning and Combinatorial Optimization IPAM Workshop A great workshop on the intersection of ML, RL, GNNs, and combinatorial optimization. Videos are available. Topics include applications of ML to chip design, TSP, physics, integer programming and more.

Top-10 Research Papers in AI A new blog post about the top-10 most cited papers in AI during the last 5 years. I looked at major AI conferences and journals (excluding CV and NLP conferences). It was quite a refreshing experience to realize that many of what we use today by default have been discovered only within the last few years. Things like Adam, Batch Norm, GCNs, etc.

MLSys 2021 Conference MLsys is one of the main conferences on applications of ML in real-world. Accepted papers for MLSys 2021 are available here. It will also feature GNN workshop and keynote speakers from NVIDIA, PyTorch, and others. Dates are April 5-9, 2021. Registration is 25$ for students.

Fresh picks from ArXiv This week on ArXiv: unsupervised community detection, reducing variance with sampling, and GNNs for quantum calculations 🧮 If I forgot to mention your paper, please shoot me a message and I will update the post. Algorithms * Graph Force Learning * Recurrent Graph Neural Network Algorithm for Unsupervised Network Community Detection * ForceNet: A Graph Neural Network for Large-Scale Quantum Calculations * Autobahn: Automorphism-based Graph Neural Nets * Graph Convolutional Embeddings for Recommender Systems SIGIR 2021 * On the Importance of Sampling in Learning Graph Convolutional Networks * Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework Software * Implementing graph neural networks with TensorFlow-Keras * Large Graph Convolutional Network Training with GPU-Oriented Data Communication Architecture Survey * Deep Graph Structure Learning for Robust Representations: A Survey