Post #852

GraphML News (July 13th) - Recursion goes brrr, Acquisition of Graphcore, Illustrated AF3 💸 Recursion and NVIDIA launched BioHive-2, a GPU cluster made of 504 H100’s which is roughly equivalent to 1 petaflops in FP16 / BF16 and perhaps sub-$50M in the costs. Some napkin math indicates it could train and fine-tune a full AlphaFold 3-like model in about 4 days. Except for ESM-3, we haven’t yet seen drug discovery models trained on such compute - congrats to Recursion, Valence, and researchers with engineers who can now really go brrr. 💸 Graphcore, a UK hardware startup offering their hardware platform (BOW IPUs), was acquired by SoftBank for rumored $500M (back in 2020 valuation was about $2.8B). Former employees likely lost their vested options ($500M is still less than $600M originally invested into the company) but let’s hope that now the future would be more stable for Graphcore and we will see more successful products. 🧬The Illustrated AlphaFold by Elana Simon and Jake Silberg from Stanford (inspired by the Illustrated Transformer) explains visually the main building blocks of the model - starting from the input data down to PairFormer, triangular attention to the diffusion module to the training losses. Things get much simpler indeed when you know which shapes are involved at each particular step. Weekend reading: Link Prediction with Untrained Message Passing Layers by Lisi Qarkaxhija, Anatol E. Wegner, and Ingo Scholtes - the unreasonable effectiveness of untrained MPNNs strikes back SE(3)-Hyena Operator for Scalable Equivariant Learning by Artem Moskalev et al - FFT with Clifford MLPs enable equivariant Hyena on long sequences up to 3.5M tokens on a single GPU On the Expressive Power of Sparse Geometric MPNNs by Yonatan Sverdlov, Nadav Dym - enabling equivariant GNNs on sparse graphs (usually EGNNs work on fully-connected graphs)