Am Neumarkt 😱

#fun A moment of joy for Friday. https://www.youtube.com/watch?v=ZI0w_pwZY3E

🤣 https://twitter.com/BaselessPursuit/status/1372205941450493955?s=20

#ML How do we interpret the capacities of the neural nets? Naively, we would represent the capacity using the number of parameters. Even for Hopfield network, Hopfield introduced the concept of capacity using entropy which in turn is related to the number of parameters. But adding layers to neural nets also introduces regularizations. It might be related to capacities of the neural nets but we do not have a clear clue. This paper introduced a new perspective using sparse approximation theory. Sparse approximation theory represents the data by encouraging parsimony. The more parameters, the more accurate the model is representing the training data. But it causes generalization issues as similar data points in the test data may have been pushed apart [^Murdock2021]. By mapping the neural nets to shallow "overcomplete frames", the capacity of the neural nets is easier to interpret. [Murdock2021]: Murdock C, Lucey S. Reframing Neural Networks: Deep Structure in Overcomplete Representations. arXiv [cs.LG]. 2021. Available: http://arxiv.org/abs/2103.05804

#fun India is growing so fast Oh Germany... Global AI Vibrancy Tool Who’s leading the global AI race? https://aiindex.stanford.edu/vibrancy/

#ML Simple algorithm, powerful results https://avinayak.github.io/algorithms/programming/2021/02/19/finding-mona-lisa-in-the-game-of-life.html

#ML hmmm https://bair.berkeley.edu/blog/2021/03/09/maxent-robust-rl/

#ML I just found an elegant decision tree visualization package for sklearn. I have been trying to explain decision tree results to many business people. It is very hard. This package makes it much easier to explain the results to a non-techinical person. https://github.com/parrt/dtreeviz

#fun By Sumit Dalmiya Source: https://www.linkedin.com/posts/sumit-dalmiya_fridayfun-datasciencecareers-activity-6773667345366880256-KXtn

#fun > Growth in data science interviews plateaued in 2020. Data science interviews only grew by 10% after previously growing by 80% year over year. > Data engineering specific interviews increased by 40% in the past year. https://www.interviewquery.com/blog-data-science-interview-report

#ML#Phyiscs The easiest method to apply constraints to a dynamical system is through Lagrange multiplier, aka, penalties in statistical learning. Penalties don't guarantee any conservation laws as they are simply penalties, unless you find the multiplers carrying some physical meaning like what we have in Boltzmann statistics. This paper explains a simple method to hardcode conservation laws in a Neural Network architecture. Paper: https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.126.098302 TLDR: See the attached figure. Basically, the hardcoded conservation is realized using additional layers after the normal neural network predictions. A quick bite of the paper: https://physics.aps.org/articles/v14/s25 Some thoughts: I like this paper. When physicists work on problems, they like dimensionlessness. This paper follows this convention. This is extremely important when you are working on a numerical problem. One should always make it dimensionless before implementing the equations in code.

Agenda for AI CON 2021

#event If you are interested in free online AI Cons, Bosch CAI is organizing the AI Con 2021. This event starts tomorrow. https://www.ubivent.com/start/AI-CON-2021