Hey there, this is the second edition of this newsletter where I share new things I find interesting in technology and AI (paper summaries, open-source, more) straight to your inbox!
My updates on Open source 🧑‍💻
Added an end-to-end training example for image classification and pre-trained models on ImageNet using this architecture to foster usability.
I got featured in the latest version of the Google DevLibrary newsletter under the contributor spotlight for being one among the top 5 DevLibrary contributors.
My writings đź“ť
In this article, I’ll give you some opinionated tips to help you build a great open-source project you can start using. You can also use these tips for building hackathon projects.
Great reads from the communityđź“–
A Transformer trained to produce all of the model parameters of a small ConvNet works really well for meta-learning! Allows you to decouple the complexity of the large task space from the complexity of individual tasks. Works very well for small target CNN architectures.
GANs are notoriously unstable to train. A widely used empirical heuristic to stabilize training is spectral normalization (SN). Zinan Lin explains why and how spectral normalization controls exploding and vanishing gradients and how you can further improve SN.
Realistic editing of real videos. Encoders are smooth at the local scale, generator tuning works great globally, and maintains alignment. Together they almost perfectly preserve the original video’s consistency. Maintains a higher degree of temporal consistency over the current state-of-the-art.
Models like GPT-3 in part are not aligned with their users. To make models safer, more helpful, and more aligned use an existing technique called reinforcement learning from human feedback (RLHF) to create InstructGPT obtaining more truthful and less toxic outputs/ Something really interesting: this has 100x fewer parameters than GPT-3!
A very interesting talk by Florian Marquardt as a part of his lecture series “Advanced Machine Learning for Physics, Science, and Artificial Scientific Discovery”.
Open-source from community đź‘Ź
Test Suites for Validating ML Models & Data. Deepchecks is a Python package for comprehensively validating your machine learning models and data with minimal effort.
A Self-Supervised Speech Pre-training and Representation Learning Toolkit.
That’s all, hope you liked this. Stay tuned for more updates. Feel free to submit any links for the next issue.
Regards,
Rishit Dagli