Post #897

GraphML News (June 21st) - Skala, Temporal RDL, Future of Graph Learning, Erwin ⚛️ MSR AI 4 Science announced Skala - an exchange-correlation (XC) ML potential to estimate chemical properties of molecules (energy and force fields). Skala represents molecules via density features obtained from meta-generalized-gradient approximation (meta-GGA) and is practically an irregular integration grid. The main model employs radial functions and spherical harmonics to capture non-local interactions and run integration over space. Skala was trained on a new dataset of 150k data points and reaches SOTA MAE on the W4-17 dataset. Preprint and data are available (lots of fancy equations in the appendix). 🕸️ Kumo published an interesting piece on temporal dependencies in relational DL where features change over time - note that in RelBench edges have timestamps but features are static. They tried predictive forecasting (training a regression head over the graph and history of features) vs generative forecasting (training a diffusion model instead) which experimentally give pretty similar results. Transformers have been used all the way (both for graph encoding and for sequence modeling). 🌟 The Graph Learning on Wednesdays (GLOW) reading group summarized a series of recent discussions on the future of graph learning in a new blog post with opinions from many renowned researchers as to why graph learning is experiencing a certain identity crisis and lack of glamour compared to LLMs, agents, and mainstream AI research. Partly, it revolves around missing “killer” applications which would attract new researchers - nobody needs another variation of GCN / GAT / GT, or some esoteric positional encodings, or yet another self-supervised loss to train on Cora when you can run true scientific discovery with new generations of LLMs and agents. Our take on the problem is in the ICML’25 position paper - find us in Vancouver to chat and share your opinions in the comments. 🎱 Maksim Zhdanov (UvA) published a nice visual introduction to ball tree attention used in the recent Erwin Transformer and how to expand receptive field to large structures in subquadratic time. Erwin is quite strong on MD and PDE modeling tasks, find out about smart tricks for sparser attention. 🎉 Finally, the GDL book now includes a new chapter 5 on Graphs - in addition to standard architectures, the chapter talks asynchronous and topological message passing, as well as looking at the transformer layer (self-attention + MLP) through the lens of message passing. The illustrations are cool - props to the renowned tikz magician Petar Veličković. Weekend reading: Don’t procrastinate with new papers, go finish those NeurIPS reviews or ACs will be shaming you and send news to your co-authors how slow you are 🙂