Post #749

Friday Graph ML News: Ankh Protein LM, Deadlines, and New Blogs ☥ This week we do see a new big model: meet Ankh, a protein LM! Thanks to the recent observation of importance of data size and training vs model size, 1.5B Ankh often outperforms 15B ESM-2B on contact prediction, structure prediction, and a good bunch of protein representation learning tasks. Arxiv pre-print is available as well! If you didn’t make it to polish the submission for ICML or IJCAI, consider other upcoming submission deadlines: - Deep Leaning for Graphs @ International Joint Conference on Neural Networks: Jan 31st - Special Issue on Graph Learning @ IEEE Transactions on Neural Networks and Learning Systems: March 1st - Graph Representation Learning track @ European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning: May 2nd The Summer Geometry Initiative (MIT) is a six-week paid summer research program introducing undergraduate and graduate students to the field of geometry processing, no prior experience is required: apply until February 15th. New articles and blogs about graphs and more general deep learning: - Quanta Magazine published a fascinating article on the discovery of the shortest path algorithm on graphs with negative edge weights; - Kexin Huang prepared a post explaining a variety of datasets available in the Therapeutic Data Commons from drug-target interactions to retrosynthesis and predicting CRISPR editing outcomes - Tim Dettmers updated his annual report on most efficient GPUs per $ with new data from H100. In relative performance, if you don’t have H100’s — get RTX 4090; for perf per $ 4070 Ti is surprisingly in the top. - Google published a Deep Learning Tuning Playbook - a collection of tuning advice that will help you to squeeze that 1% of performance and get top-1 in OGB! - Finally, a huge post from Lilian Weng on optimizing inference of large Transformers