Am Neumarkt 😱

确实有很多，比如我用 ack 替代了 grep，速度快了不少。 https://www.ruanyifeng.com/blog/2022/01/cli-alternative-tools.html

#ml https://ruder.io/ml-highlights-2021/

#visualization Seaborn is getting a new interface. Would be great if the author defines a dunder method _ _ add _ _ () instead of using .add() method. Using dunder add, we can simply use + on layers. Nevertheless, we can all move away from plotnine when the migration is done. https://seaborn.pydata.org/nextgen/

#ds Deepnote supports Great Expectations (GE) now. I ran their template notebook: https://deepnote.com/project/Reduce-Pipeline-Debt-With-Great-Expectations-mLT9DFCQSpW4kUBAzzdhBw/%2Fnotebook.ipynb/#00000-e170fae0-7e06-4a7a-85f3-343584ec4b94

#visualization Beautiful, elegant, and informative. It reminds me of the Netflix movie chromatic storytelling visualization. Full image: https://zenodo.org/record/5828349 Other discussions: https://www.reddit.com/r/dataisbeautiful/comments/s6vh8k/dutch_astronomer_cees_bassa_took_a_photo_of_the/

#python I thought it was a trivial talk in the beginning. But I quickly realized that I may know every each piece of the code mentioned in the video but the philosophy is what makes it exciting. He talked about some fundamental ideas of Python, e.g., protocols. After watching this video, an idea came to me. Pytorch lightning has implanted a lot of hooks in a very pythonic way. This is what makes pytorch lightning easy to use. (So if you do a lot of machine learning experiments, pytorch lightning is worth a try.) https://youtu.be/cKPlPJyQrt4

#ds https://2022.pycon.de/blog/pyconde-pydata-berlin-tickets/

#data#ds Disclaimer: I'm no expert in state diagram nor statecharts. It might be something trivial but I find this useful: Combined with some techniques in statecharts (something frontend people like a lot), state diagram is a great way to document what our data is going through in data (pre)processing. For complicated data transformations, we can make the corresponding state diagram and follow your code to make sure it is working as expected. The only thing is that we are focusing on the state of data not any other system. We can use some techniques from statecharts, such as hierarchies and parallels. State diagram is better than flowchart in this scenario because we are more interested in the different states of the data. State diagrams automatically highlights the states and we can easily spot the relevant part in the diagram and we don’t have to start from the beginning. I documented some data transformations using state diagrams already. I haven't tired but it might also help us document our ML models. References: 1. https://statecharts.dev 2. https://en.wikipedia.org/wiki/State_diagram

#visualization Pu X, Kay M. A probabilistic grammar of graphics. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. New York, NY, USA: ACM; 2020. doi:10.1145/3313831.3376466 Available at: https://dl.acm.org/doi/10.1145/3313831.3376466 A very good read if you are visualizing probability densities a lot. The paper began with a common mistake people make when visualizing densities. Then they proposed a systematic grammar of graphics for probabilities. They also provide a package (quite preliminary, see here https://github.com/MUCollective/pgog ).

#ml#science I remember several years ago when I was still doing my PhD, there's this contest about predicting protein structure and none of them was working well. At that time, I would never have thought we could have anything like AlphaFold in a few years. . https://www.science.org/content/article/breakthrough-2021

#visualization#fun https://www.githubwrapped.com/

#ML#Transformers Alammar J. The Illustrated Transformer. [cited 14 Dec 2021]. Available: http://jalammar.github.io/illustrated-transformer/ So good.