Post #843

GraphML News (June 1st) - GNNs for Automotive Vision, NeurIPS submissions A fresh example of applying GNNs in real-world problems is provided in the Nature paper Low-latency automotive vision with event cameras by Daniel Gehrig and Davide Scaramuzza from Uni Zurich. There, GNNs help to parse temporal events (like appearance of a pedestrian on a road) and save a lot of compute by updating only local neighborhood of changed patches. The model (with efficient CUDA implementation) works in real time in cars! Code and video demo are available. The week brought a handful of cool new papers formatted with the NeurIPS template (what could that mean 🤔) - let’s see: 🧬Genie 2 by AlQuraishi lab - better protein diffusion model now supporting multi-motif scaffolding, outperforms RFDiffusion, FrameFlow, and Chroma, code is available. 🦄LoGAH: Predicting 774-Million-Parameter Transformers using Graph HyperNetworks with 1/100 Parameters - Xinyu Zhou feat. Boris Knyzev - the next iteration of the Graph Hypernetwork (GHN-3) that directly predicts parameters of neural networks, now with an efficient module for transformer-sized matrices. The model can predict weights of GPT-2 and ViT-sized networks! Code 🍭Understanding Transformer Reasoning Capabilities via Graph Algorithms by Clayton Sanford and Google team feat. Anton Tsitsulin and Bryan Perozzi - a theoretical study on transformers and their ability to solve graph problems. The study reveals that, eg, depth has to scale as O(log(V+E)) from the graph size for parallelizable problems, and additional scaling of width for search problems. Besides, there is a comparison between GNNs and Transformers (trained from scratch and fine-tuned T5) on the GraphQA benchmark. Prompting LLMs doesn’t really work. 🤓 Two papers on flow matching from Michael Bronstein’s lab: Fisher Flow Matching for Generative Modeling over Discrete Data by Oscar Davis et al - the best discrete FM model so far, and Metric Flow Matching for Smooth Interpolations on the Data Manifold by Kacper Kapusniak et al - improvement of the OT-CFM (conditional flow matching with optimal transport). We’ll be posting more new cool papers in the coming days!