Post #719

Blog Posts of the Week A few fresh blog posts to add to your weekend reading list. Graph Neural Networks as gradient flows by Michael Bronstein, Francesco Di Giovanni, James Rowbottom, Ben Chamberlain, and Thomas Markovich. The blog summarizes recent efforts in understanding GNNs from the physics perspective. Particularly, the post describes how GNNs can be seen as gradient flows that help in heterophilic graphs. Essentially, the approach implies having one symmetric weight matrix W shared among all GNN layers, residual connections, and non-linearities can be dropped. Under this sauce, classic GCNs by Kipf & Welling strike back! Graph-based nearest neighbor search by Liudmila Prokhorenkova and Dmitry Baranchuk. The post gives a nice intro to the graph-based technology (eg, HNSW) behind many vector search engines and reviews recent efforts in improving scalability and recall. Particularly, the authors show that non-Euclidean hyperbolic space might have a few cool benefits unattainable by classic Euclidean-only algorithms. Long Range Graph Benchmark by Vijay Dwivedi. Covered in one the previous posts in this channel, the post introduces a new suite of tasks designed for capturing long-range interactions in graphs. Foundation Models are Entering their Data-Centric Era by Chris Ré and Simran Arora. The article is very relevant to any large-scale model pre-training in any domain, be it NLP, Vision, or Graph ML. The authors observe that in the era of foundation models we have to rethink how we train such big models, and data diversity becomes the single most important factor of inference capabilities of those models. Two lessons learned by the authors: “Once a technology stabilizes the pendulum for value swings back to the data” and “We can (and need to) handle noise”.