Graph Machine Learning

ICLR 2021 Graph Papers Last Friday submissions to ICLR 2021 became available for reading. There are 3013 submissions, about 210 graph papers (7% of total). About every third paper came from rejection of NeurIPS (which is based on overlap of paper submissions), which surprised me not just on sheer volume, but also because I'm puzzled where the remaining 6000 rejected papers are resubmitted to. I extracted graph papers, which are attached, and categorized them loosely in 4 topics: model, theory, application, and survey. Most of the papers (171) are about new models (general GNNs, graph models for new problems, improvements over existing models). 22 papers are novel applications in physics, chemistry, biology, etc. 13 are theoretical papers, and 4 are surveys/evaluation benchmarks.

The next big thing: the use of graph neural networks to discover particles It's great to see that GNNs can be useful for fundamental applications such as new particles discovery. In another post by Fermilab, US-based physics lab, researchers discuss that they are able to move GNNs to production for Large Hadron Collider (LHC) at CERN. The goal is to process millions of images and select those that could be relevant to discovery of new particles. They expect to see the results in LHC's Run 3 in 2021. ArXiv preprint is available online.

NYC Deep Learning Course: Structured Prediction Final lecture of the course on deep learning led by Yann LeCun. It covers structured prediction, energy-based factor graphs, and graph transformer networks.

SE(3)-Transformers A blog post about a recent paper (NeurIPS 2020) that introduces group theory to set functions. It seems like it performs on par with state-of-the-art methods for classification and regression, but at least is provably equivariant.

Fresh picks from ArXiv Many papers caught my attention this week (and it's not because of NeurIPS), very interesting stuff: debunking value of scene graphs, extrapolation of GNNs, GraphNorm, Alibaba KG construction, closed formulas for graphlets, and applications to river dynamics 🌊 If I forgot to mention your paper, please shoot me a message and I will update the post. Conferences - Are scene graphs good enough to improve Image Captioning? AACL 2020 - Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph EMNLP 2020 - Structure Aware Negative Sampling in Knowledge Graphs EMNLP 2020 with William L. Hamilton - Message Passing for Hyper-Relational Knowledge Graphs EMNLP 2020 with Michael Galkin - Sub-graph Contrast for Scalable Self-Supervised Graph Representation Learning ICDM 2020 -Graph neural induction of value iteration GRL+ 2020 - Heterogeneous Molecular Graph Neural Networks for Predicting Molecule Properties ICDM 2020 GNN - How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks with Stefanie Jegelka - Learning Graph Normalization for Graph Neural Networks Applications - Physics-Guided Recurrent Graph Networks for Predicting Flow and Temperature in River Networks - SIA-GCN: A Spatial Information Aware Graph Neural Network with 2D Convolutions for Hand Pose Estimation Industry - AliMe KG: Domain Knowledge Graph Construction and Application in E-commerce Math - Counting five-node subgraphs Survey - A survey of graph burning

NeurIPS 2020 stats Dates: Dec 6 - 12 Where: Online Price: $25/$100 (students/non-students) • 9454 submissions (vs 6743 in 2019) • 1900 accepted (vs 1428 in 2019) • 20.1% acceptance rate (vs 21% in 2019) • 123 graph papers (6.5% of total)

Graph Machine Learning research groups: Alejandro Ribeiro I do a series of posts on the groups in graph research, previous post is here. The 15th is Alejandro Ribeiro, head of Alelab at UPenn and the leading author of the ongoing GNN course. Alejandro Ribeiro (1975) - Affiliation: University of Pennsylvania - Education: Ph.D. in University of Minnesota in 2006 (advisor: Georgios B. Giannakis) - h-index 51 - Awards: Hugo Schuck best paper award, paper awards at CDC, ACC, ICASSP, Lindback award, NSF award - Interests: wireless autonomous networks, machine learning on network data, distributed collaborative learning

Message Passing for Hyper-Relational Knowledge Graphs This is a guest post by Michael Galkin about their recently accepted paper at EMNLP. Traditionally, knowledge graphs (KGs) use triples to encode their facts, eg subject, predicate, object Albert Einstein, educated at, ETH Zurich Simple and straighforward, triple-based KG are extensively used in a plethora of NLP and CV tasks. But can triples effectively encode richer facts when we need them? If we have the two facts: Albert Einstein, educated at, ETH Zurich Albert Einstein, educated at, University of Zurich what can we say about Einstein's education? Did he attend two universities at the same time? 🤨 It is a common problem of triple-based KGs when we want to assign more attributes to each typed edge. Luckily, the KG community has two good ways to do that: with RDF* and Labeled Property Graphs (LPGs). With RDF* we could instantiate each fact with qualifiers: ( Albert_Einstein educated_at ETH_Zurich ) academic_degree Bachelor ; academic_major Maths . ( Albert_Einstein educated_at University_of_Zurich ) academic_degree Doctorate ; academic_major Physics. We call such KGs as hyper-relational KGs. Wikidata follows the same model, here is Einstein's page where you'd find statements (hyper-relational facts) with qualifiers (those additional key-value edge attributes). Interestingly, there is pretty much nothing 🕳 in the Graph ML field for hyper-relational graphs. We have a bunch of GNN encoders for directed, multi-relational, triple-based KGs (like R-GCN or CompGCN), and nothing for hyper-relational ones. In our new paper, we design StarE ⭐️, a GNN encoder for hyper-relational KGs (like RDF* or LPG) where each edge might have unlimited amount of qualifier pairs (relation, entity). Moreover, those entities and relations do not need to be qualifier-specific, they can be used in the main triples as well! In addition, we carefully constructed WD50K, a new Wikidata-based dataset for link predicion on hyper-relational KGs, and its 3 decendants for various setups. Experiments show that qualifiers greatly improve subject/object prediction accuracy, sometimes reaching a whopping 25 MRR points gap. More applications and tasks are to appear in the future work! Paper: https://arxiv.org/abs/2009.10847 Blog: Medium friends link Code: Github

3DGV Seminar: Michael Bronstein There is a good ongoing seminar on 3D geometry and vision. Last seminar was presented by Michael Bronstein who was talking about inductive biases, timeline of GNN architectures, and several successful applications. Quite insightful.