Graph Machine Learning

Graph Machine Learning Newsletter I finally had time to compose the first issue for my newsletter on graph machine learning. I will be out soon! Please subscribe and share it with your friends: http://newsletter.ivanovml.com/ (or, in case, it gives a warning here is a backup link: https://www.getrevue.co/profile/graphML). My hope it will be similar to Ruder's newsletter on NLP, highlighting recent developments, current trends, and upcoming events in GML. I plan to send 1-2 issues per month, so it will be less frequent but more long read about our field. In case you saw recent blog posts, interviews, conference highlights, industry updates, or anything else worth sharing with a community, don't hesitate to write to me.

Controlling Fake News using Graphs and Statistics This is a guest post by Siddharth Bhatia about their recent work with Christos Faloutsos on anomaly detection in streaming data. MIDAS, Microcluster-Based Detector of Anomalies in Edge Streams (AAAI 2020) uses unsupervised learning to detect anomalies in a streaming manner in real-time. It was designed keeping in mind the way recent sophisticated attacks occur. MIDAS can be used to detect intrusions, Denial of Service (DoS), Distributed Denial of Service (DDoS) attacks, financial fraud and fake ratings. MIDAS combines a chi-squared goodness-of-fit test with the Count-Min-Sketch (CMS) streaming data structures to get an anomaly score for each edge. It then incorporates temporal and spatial relations to achieve better performance. MIDAS provides theoretical guarantees on the false positives and is three orders of magnitude faster than existing state of the art solutions. Paper: https://arxiv.org/abs/1911.04464 Code: https://github.com/Stream-AD/MIDAS

Graph Machine Learning research groups: Kristian Kersting I do a series of posts on the groups in graph research, previous post is here. The 11th is Kristian Kersting, co-author of TU data set and several graph kernels. Kristian Kersting (1973) - Affiliation: TU Darmstadt - Education: Ph.D. at the University of Freiburg, Germany in 2014 (supervised by Luc De Raedt); - h-index: 49; - Awards: best paper at ECML, AAAI; Inaugural German AI Award; - Interests: graph kernels, graph data sets

Main theme from GRL+ workshop I already mentioned it, but let me add more things about trends in GML (credits to Petar Veličković). The biggest theme from GRL+ workshop was the explicit consideration of structure, which so far was largely ignored in GNNs (i.e. one would just assume a given graph without thinking how it got there or whether it could be specialized for the task at hand). In the accepted papers, we have many works which tackle latent structure inference (e.g. Differentiable Graph Module, set2graph, Relate-and-Predict, GFSA, and our PGN are all examples thereof) and also works which attempt to explicitly exploit structure in the data for prediction (e.g. the recent subgraph isomorphism counting paper). This direction was echoed a lot in our invited talks as well. Thomas Kipf was talking about relational structure discovery (NRI, CompILE and his recent slot attention work). Kyle Cranmer was talking about how critical structure discovery is in physics-based applications and inductive biases, highlighting especially his set2graph work as well as their recent work on discovering symbolic representations. Danai Koutra talking how graphs can be appropriately summarized and how to design GNN layers to deal with heterophily. Tina Eliassi-Rad gave an amazing lecture-style talk on how topology and structure can be leveraged in machine learning more generally. During our Q&A session, she was asked to give comments on the explosive usage datasets like Cora (as she is one of the authors on the paper that originally proposed Cora, Citeseer etc). And she made a very important 'wakeup call' to GRL folks that we shouldn't think our graphs fall from the sky, and on the topic of using heavy-duty GNN methods and hyperbolic embeddings, etc in the real world, we should always ask the question: 'do we really expect our graphs to be coming from a distribution like this?'. The videos with all of it should be available in the coming weeks.

Podcast with Michael Bronstein There is a podcast called This Week in Machine Learning & AI (TWIML) about aspects of AI. Michael Bronstein, head of graph machine learning at Twitter, gave recently a lengthy interview talking about evolution of the field over the last 2 years. He describes current challenges (e.g. scalability), difference between industrial and academic settings for graphs, his recent works as well as prediction of where the area of GML is moving towards.

Discovering Symbolic Models in Physical Systems Using Deep Learning Today (July 29, at 12:00 EDT) will be a zoom lecture about applying GNN to cosmology by Shirley Ho at Physics ∩ ML seminar. Abstract: We develop a general approach to distill symbolic representations of a learned deep model by introducing strong inductive biases. We focus on Graph Neural Networks (GNNs). The technique works as follows: we first encourage sparse latent representations when we train a GNN in a supervised setting, then we apply symbolic regression to components of the learned model to extract explicit physical relations. We find the correct known equations, including force laws and Hamiltonians, can be extracted from the neural network. We then apply our method to a non-trivial cosmology example—a detailed dark matter simulation—and discover a new analytic formula that can predict the concentration of dark matter from the mass distribution of nearby cosmic structures. The symbolic expressions extracted from the GNN using our technique also generalized to out-of-distribution-data better than the GNN itself. Our approach offers alternative directions for interpreting neural networks and discovering novel physical principles from the representations they learn.

Fresh picks from ArXiv This week highlights a new knowledge graph about covid-19, applications to program similarity and drug discovery as well as a group of accepted papers to ECCV 20. GNN • COVID-19 Knowledge Graph: Accelerating Information Retrieval and Discovery for Scientific Literature • funcGNN: A Graph Neural Network Approach to Program Similarity • The expressive power of kth-order invariant graph networks • Fast Graphlet Transform of Sparse Graphs • Visualizing Deep Graph Generative Models for Drug Discovery • Second-Order Pooling for Graph Neural Networks • Graph-PCNN: Two Stage Human Pose Estimation with Graph Pose Refinement • Hierachial Protein Function Prediction with Tails-GNNs with Petar Veličković Conferences • Multi-view adaptive graph convolutions for graph classification ECCV 20 • Comprehensive Image Captioning via Scene Graph Decomposition ECCV 20 • Differentiable Hierarchical Graph Grouping for Multi-Person Pose Estimation ECCV 20 • Grale: Designing Networks for Graph Learning with Bryan Perozzi, KDD 20 • Edge-aware Graph Representation Learning and Reasoning for Face Parsing ECCV 20 Surveys • A Survey on Complex Question Answering over Knowledge Base: Recent Advances and Challenges • A Survey on Graph Neural Networks for Knowledge Graph Completion

Trends in GML I think GRL+ workshop is really cool: it gathers people in GML and discusses ideas that are not fully developed but will be soon. It's like peeking into the crystal ball. Petar Veličković, one of the organizers of this workshop, outlined the following trends: - Emerging work on performance / scalability (e.g. SIGN, Weisfeiler & Leman go sparse) - KG embeddings are as strong as ever (e.g. neural multi-hop reasoning, MPQE, Stay Positive, UniKER) - proposal of many datasets/benchmarks/libraries (Wiki-CS, TUDataset, Spektral, Graphein, Geo2DR, Geoopt) - work on computational chemistry (with applications to drug design/repurposing), such as the Retrosynthesis paper (which won best paper award) - Applications of GRL for algorithmic reasoning (e.g. Neural Bipartite Matching, planning with neuro-algorithmic policies. and PGNs) But the obvious standout, not only in the papers but also in most of our invited talks, is the explicit consideration of structure.

Opening slides from GRL+ workshop (ICML 20) by Petar Veličković.

IJCAI 2020 stats IJCAI moved its dates to Jan 2021. Dates: Jan 2021 Where: Japan/Online Link to papers • 4717 submissions (vs 4752 in 2019) • 592 accepted (vs 850 in 2019) • 12.6% acceptance rate (vs 17.9% in 2018) • 55 graph papers

ECCV 2020 stats ECCV is among the best conferences in computer vision. Dates: Aug 23-28 Where: Online Cost: £150 Link to papers • 5025 submissions (vs 2439 in 2019) • 1361 accepted (vs 776 in 2019) • 27.1% acceptance rate (vs 31.8% in 2018) • 4 graph papers