Найти похожее

#ml In his MinT paper, Hyndman said he confused these two quantities in his previous paper. 😂 MinT is a simple method to make forecasts with hierarchical structure coherent. Here coherent means the sum of the lower level forecasts equals the higher level forecasts. For example, our time series has a strucutre like sales of coca cola + sales of spirit = sales of beverages. If this relations holds for our forecasts, we have coherent forecasts. This may sound trivial, the problem is in fact hard. There are many trivial methods such as only forecasting lower levels (coca cola, spirit) then use the sum as the higher level (sales of beverages). These are usually too naive to be effective. MinT is a reconciliation method that combines high level forecasts and the lower level forecasts to find an optimal combination/reconciliation. https://robjhyndman.com/papers/MinT.pdf

#ml This is amazing compiled_model = torch.compile(model) https://pytorch.org/get-started/pytorch-2.0/

#ml You spent 10k euros on GPU then realized the statistical baseline model is better. 🤣 https://github.com/Nixtla/statsforecast/tree/main/experiments/m3

#ml Diffusion Models for Handwriting Generation https://github.com/tcl9876/Diffusion-Handwriting-Generation

#ml https://arxiv.org/abs/2210.10101v1 Bernstein, Jeremy. 2022. “Optimisation & Generalisation in Networks of Neurons.” ArXiv [Cs.NE], October. https://doi.org/10.48550/ARXIV.2210.10101.

#ml 😱😱😱 Fancy Video: https://fb.watch/giO0tV4N4T Press: https://research.facebook.com/publications/dressing-avatars-deep-photorealistic-appearance-for-physically-simulated-clothing/ Xiang, Donglai, Timur Bagautdinov, Tuur Stuyck, Fabian Prada, Javier Romero, Weipeng Xu, Shunsuke Saito, et al. 2022. “Dressing Avatars: Deep Photorealistic Appearance for Physically Simulated Clothing.” ArXiv [Cs.GR], June. https://arxiv.org/abs/2206.15470

#ml https://developer.nvidia.com/blog/how-optimize-data-transfers-cuda-cc/ I find this post very useful. I have always wondered what happens after my dataloader prepared everything for the GPU. I didn’t know that CUDA has to copy the data again to create page-locked memory. I used to set pin_memory=True in a PyTorch DataLoader and benchmark it. To be honest, I have only observed very small improvements in most of my experiments. So I stopped caring about pin_memory. After some digging, I also realized that performance from setting pin_memory=True in DataLoader is ticky. If we don’t use multiprocessing nor reuse the page-locked memory, it is hard to expect any performance gain. (some other notes: https://datumorphism.leima.is/cards/machine-learning/practice/cuda-memory/)

#ml Amazon has been updating their Machine Learning University website. It is getting more and more interesting. They have added an article about linear regression recently. There is a section in this article about interpreting linear models and it is just fun. https://mlu-explain.github.io/ ( Time machine: https://t.me/amneumarkt/293 )

#ML This is interesting. Toy Models of Superposition. [cited 15 Sep 2022]. Available: https://transformer-circuits.pub/2022/toy_model/index.html#learning

#ml https://ai.googleblog.com/2022/08/optformer-towards-universal.html?m=1 I find this work counter intuitive. They took some descriptions of the optimization in machine learning and trained a transformer to "guesstimate" the hyperparameters of a model. I understand that human being has some "feeling" of the hyperparameters after working with the data and model for a while. But it is usually hard to extrapolate such knowledge when we have completely new data and models. I guess our brain is doing some statistics based on our historical experiments. And we call this intuition. My "intuition" is that there is little generalizable knowledge in this problem. 🙈 It would have been so great if they investigated the saliency maps.

#ml Fotios Petropoulos initiated the forecasting encyclopaedia project. They published this paper recently. Petropoulos, Fotios, Daniele Apiletti, Vassilios Assimakopoulos, Mohamed Zied Babai, Devon K. Barrow, Souhaib Ben Taieb, Christoph Bergmeir, et al. 2022. “Forecasting: Theory and Practice.” International Journal of Forecasting 38 (3): 705–871. https://www.sciencedirect.com/science/article/pii/S0169207021001758 Also available here: https://forecasting-encyclopedia.com/ The paper covers many recent advances in forecasting, including deep learning models. There are some important topics missing but I’m sure they will cover them in future releases.

#ml https://arxiv.org/abs/2205.02302 Kreuzberger D, Kühl N, Hirschl S. Machine Learning Operations (MLOps): Overview, definition, and architecture. arXiv [csLG]. 2022 [cited 17 Jul 2022]. doi:10.48550/ARXIV.2205.02302