Post #859

GraphML News (August 24th) - Psiformer, ML potentials arena, Single-cell foundation models ⚛️ DeepMind announced the updated version of Psiformer (together with the paper in Science, twitter thread, and source code in Jax) - a transformer for quantum physics tasks. The new model can approximate excited states of molecules on par or better than existing gold standard models. Excited energy states are responsible for lasers, semiconductors, solar panels, fluorescence, and many other phenomena - a huge potential for Psiformer in industrial applications. 🏆 Continuing with energy states - you probably know that the ultimate LLM benchmark those days is the ELO rating on the Chatbot Arena. Yuan Chiang started a similar effort for ML potential models (MLIP Arena) featuring 3 tasks: two atoms of the same type (the only LB for now) and two molecular dynamics tasks (loading time is slow). The supported models for now are Equiformer V2, CHGNet, MACE MP, M3GNet, SevenNet, and the GPAW DFT baseline from the DFT world. 🎻 Single-cell foundation models are getting more attention. The new scCello by Mila is a transformer trained on the masked LM task together with the alignment loss using the Cell Ontology. scCello in the zero-shot inference regime outperforms end-to-end trained models on tasks like cell type classification, marker gene prediction, and batch integration. If you are interested to learn more, have a look at the fresh survey on transformers in SC omics. Weekend reading: more foundation models and materials science: A foundation model for clinician-centered drug repurposing by Kexin Huang et al feat. Jure Leskovec and Marinka Zitnik - introduces TxGNN, a graph foundation model for drug repurposing trained on a medical KG of 17k diseases and 8k drugs, strong zero-shot performance included. The model and example weights are already on Github. Microsoft published the source code of Aurora - FM for atmospheric forecasting, consists of Perceiver encoder/decoder and SwinTransformer as the backbone. Crystalline Material Discovery in the Era of Artificial Intelligence by Zhenzhong Wang et al (thanks to Wanyu Lin for highlighting the work) - a survey on predictive and generative models for crystals, with the github repo of relevant papers From Text to Insight: Large Language Models for Materials Science Data Extraction + tutorial online book by Mara Schilling-Wilhelmi, Martiño Ríos-García et al. LLMs are surprisingly strong in generating 3D structures of solid-state materials (ICLR 2024) on par with fancy equivariant diffusion models, this survey studies how much MatSci data LLMs could possibly feed.