Am Neumarkt 😱

#ML Duan T, Avati A, Ding DY, Thai KK, Basu S, Ng AY, et al. NGBoost: Natural Gradient Boosting for probabilistic prediction. arXiv [cs.LG]. 2019. Available: http://arxiv.org/abs/1910.03225 (I had it on my reading list for a long time. However, I didn't read it until today because the title and abstract are not attractive at all.) But this is a good paper. It goes deep to dig out the fundamental reasons why some methods work and others don't. When inferring probability distributions, it is straightforward to come up with methods with parametrized distributions (statistical manifolds). Then, by tuning the parameters, we adjust the distribution to fit our dataset the best. The problem is the choice of the objective function and optimization methods. This paper mentioned a most generic objective function and a framework to optimize the model along the natural gradient instead of just the gradient w.r.t. the parameters. Different parametrizations of the objective is like coordinate transformations and chain rule only works if the transformations are in a "flat" space but such "flat" space is not necessarily a good choice for a high dimensional problem. For a space that is approximately flat in a small region, we can define distance like what we do in differential geometry[^1]. Meanwhile, just like "covariant derivatives" in differential geometry, some kind of covariant derivative can be found on statistical manifolds and they are called "natural derivatives". Descending in the direction of natural derivatives is navigating the landscape more efficiently. [^1]: This a Riemannian space

#visualization#art#fun More like a blog post… But the visualisation is cool. I posted it as a comment. [2109.15079] Asimov's Foundation -- turning a data story into an NFT artwork https://arxiv.org/abs/2109.15079

#data Finally. Announcing Streamlit 1.0! 🎈 https://blog.streamlit.io/announcing-streamlit-1-0/

#academia This is not only Julia for biologists. It is for everyone who is not using Julia. Roesch, Elisabeth, Joe G. Greener, Adam L. MacLean, Huda Nassar, Christopher Rackauckas, Timothy E. Holy, and Michael P. H. Stumpf. 2021. “Julia for Biologists.” ArXiv [q-Bio.QM]. arXiv. http://arxiv.org/abs/2109.09973.

#visualization I like this. I was testing visualization using antv's G6. It is not for data analysis as it is quite tedious to generate visualizations. Observable's plot is a much easier fluent package for data analysis. https://github.com/observablehq/plot

#visualization Neural Networks visualized in 3D Source: https://youtu.be/3JQ3hYko51Y

#career Comment: Same for many competitive careers Beware survivorship bias in advice on science careers https://www.nature.com/articles/d41586-021-02634-z

#ML scikit learn reached 1.0. Nothing exciting about these new stuff but the major release probably means something. Release Highlights for scikit-learn 1.0 — scikit-learn 1.0 documentation http://scikit-learn.org/stable/auto_examples/release_highlights/plot_release_highlights_1_0_0.html

#ML#fun I read about the story of using tensorflow in google translate [^Pointer2019]. > … Google Translate. Originally, the code that handled translation was a weighty 500,000 lines of code. The new, TensorFlow-based system has approximately 500, and it performs better than the old method. This is crazy. Think about the maintenance of the code. A single person easily maintains 500 lines of code. 500,000 lines? No way. Reference: [^Pointer2019]: Pointer I. Programming PyTorch for Deep Learning: Creating and Deploying Deep Learning Applications. O’Reilly Media; 2019.

#ML Phys. Rev. X 11, 031059 (2021) - Statistical Mechanics of Deep Linear Neural Networks: The Backpropagating Kernel Renormalization https://journals.aps.org/prx/abstract/10.1103/PhysRevX.11.031059

#visualization The Doomsday Datavisualizations - Bulletin of the Atomic Scientists https://thebulletin.org/doomsday-clock/datavisualizations/

#data https://github.com/hosseinmoein/DataFrame