Day #235 - PyTorch developer conference part 1

Session #1 - Engineering Practices for Software 2.0
Key Lessons

• New Programming Paradigm for Neural Networks
• SGD writes code in weights of neural network
• Tune Dataset, Tune model architecture, Tune the optimization
• NN in Tesla for Autopilot

Best Practices for 2.0 Stack

• Test Driven Development Workflow - Test set manually created, clean, Carefully curated test set
• CI Workflow - Automate build - Unit Tests - Automate Deployment
• Dataset is part of code - Automate Neural Network Training Jobs - Compile into Weights - Automate Deployments
• Timestamp your data
• Mono-repos in practice

Session #2 - Applied Deep Learning
Key Lessons

• Many Research Projects use PyTorch
• Pytorch - Simple, Extensible, Fast

Projects

• Deep Learning SuperSampling - New GPU, Realtime better graphics
• NN for super resolution
• DL for real time graphics
• Inpainting. http://research.nvidia.com/inpainting
• Image and Video Synthesis - https://github.com/NVIDIA/vid2vid, Create videos with temporal consistency
• Frame prediction, Optical flow, Historical data, Predict Sampling Kernel
• Wavenet - Model for generating audio samples
• Pytorch extension Apex for mix precision training

Session #3 - NLP Transfer Learning
Key Lessons

• Making more general NLP Systems
• Related tasks tend to help each other
• Decanlp.com

NLP Projects

• Question Answering
• Machine Translation
• Summarization
• Sentiment Classification
• Semantic Role Labeling
• Semantic Parsing
• Commonsense Reasoning

Techniques

• Transfer Learning
• Weight Sharing
• Zero Shot Learning
• Data Augmentation
• Domain Adaptation
• Multi-task learning

Approach

• Seq2seq model
• Classification, Extraction, Generation
• Domain Adaption
• Some ZeroShot

Sesson #4 - Deep Universal Probablistic Programming
Key Lessons

• Pyro - Probablistic Programming Language
• Modern Bayesian ML methods
• NN for modelling and inference
• Universal, Scalable, Flexible and minimal
• 3 Layer Architecture with Probablistic Programming interface
• Inference Algo on top of library
• Stochastic Variational Inference 

To be continued from 00:55:00 rest of Session


Happy Mastering DL!!!