Graph Machine Learning

Graphormer - Do Transformers Really Perform Bad for Graph Representation? | Paper Explained A nice explanation by Aleksa Gordić of the recent paper that shows how enriching node features with some structural information from the graph can help Transformer model to achieve SOTA results on OGB datasets.

Results of OGB large-scale challenge OGB team announced the results of KDD 2021 cup challenge where teams competed in node classification, triplet prediction, and graph regression tasks. Short summaries are provided for the winning solutions and it's quite interesting to see the diversity of the proposed methods: some used ensembles of GNNs, some pretrained graph embeddings, some label propagation, among others. Notably, Baidu and DeepMind scored really well on these tasks. Congrats to the winners!

Dynamic GNNs videos A new YouTube channel that discusses spatio-temporal and dynamic GNNs in an easy and fun manner.

Deep Learning DIY course A very good deep learning course by Marc Lelarge that among other things cover graph ML: graph embeddings, signal processing, and GNNs. It comes with videos, slides, notebooks, and assignments.

Fresh picks from ArXiv This week on ArXiv: analysis of transformers, resolving scalability, and new attacks ⚔️ If I forgot to mention your paper, please shoot me a message and I will update the post. Embeddings * Self-supervised Graph-level Representation Learning with Local and Global Structure with Jian Tang * Do Transformers Really Perform Bad for Graph Representation? * Order Matters: Probabilistic Modeling of Node Sequence for Graph Generation ICML 2021 * Symmetric Spaces for Graph Embeddings: A Finsler-Riemannian Approach ICML 2021 GNNs * TDGIA:Effective Injection Attacks on Graph Neural Networks KDD 2021 * Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Prediction with Jian Tang * Is Homophily a Necessity for Graph Neural Networks? * Learning to Pool in Graph Neural Networks for Extrapolation * GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddings with Jure Leskovec * Scaling Up Graph Neural Networks Via Graph Coarsening * Rethinking Graph Transformers with Spectral Attention with William L. Hamilton * Breaking the Limit of Graph Neural Networks by Improving the Assortativity of Graphs with Local Mixing Patterns * Breaking the Limits of Message Passing Graph Neural Networks Survey * Survey of Image Based Graph Neural Networks * Graph Neural Networks for Natural Language Processing: A Survey

GML Express: keynotes at ICLR, topics at ICML 2021, and new GNN tutorials. The most interesting events in graph ML during the last 2 months are in my new issue of graph ML newsletter.

PyTorch-Geometric Tutorial Talk Today, I will speak about our ICLR work "Boost then Convolve: Gradient Boosting Meets Graph Neural Networks". If you want to learn more about how GBDT and GNN work, and how they can be applied successfully for node prediction tasks, please join here at 15 (Paris time).

Graphs at ICLR 2021 Very good digest of a few graph papers at ICLR 2021. Talks about new GNNS to solve overmoothing, over-squashing, heterophily, and attention problems.

Deep Learning on Graphs for Natural Language Processing Interesting tutorial at NAACL 2021 about applications of graph models to NLP tasks such as text classification, semantic parsing, machine translation, and more. It's based on Graph4NLP library and the slides are available here.

Udemy Graph Neural Network course Online course at Udemy that covers the basics of representation learning on graphs (e.g. DeepWalk, node2vec) and popular GNN architectures, plus some PyG implementations.

Graph Neural Networking Challenge 2021 An interesting competition, organized by Technical University of Catalonia (UPC) and ITU, about building GNNs to predict source-destination routing time. The goal is to test generalization abilities of GNNs: training on small graphs and testing on much larger graphs.

Fresh picks from ArXiv This week on ArXiv: self-supervised approach without negatives, review of generative models, and semantic search at AliBaba 👞 If I forgot to mention your paper, please shoot me a message and I will update the post. GNNs * Neural message passing for joint paratope-epitope prediction with Petar Veličković * Graph Infomax Adversarial Learning for Treatment Effect Estimation with Networked Observational Data KDD 21 * GraphMI: Extracting Private Graph Data from Graph Neural Networks IJCAI 21 * Graph Barlow Twins: A self-supervised representation learning framework for graphs * Motif Prediction with Graph Neural Networks * SpreadGNN: Serverless Multi-task Federated Learning for Graph Neural Networks Algorithms * AliCG: Fine-grained and Evolvable Conceptual Graph Construction for Semantic Search at Alibaba KDD 2021 * Stochastic Iterative Graph Matching ICML 2021 * Convergent Graph Solvers Survey * Evaluation Metrics for Graph Generative Models: Problems, Pitfalls, and Practical Solutions with Karsten Borgwardt * Laplacian-Based Dimensionality Reduction Including Spectral Clustering, Laplacian Eigenmap, Locality Preserving Projection, Graph Embedding, and Diffusion Map: Tutorial and Survey * Graph-based Deep Learning for Communication Networks: A Survey