Day #220 - Pytorch (1-4 - 10 Mins Sessions)

Key Summary
Session #1

• Tensors - Data Structures of Deep Learning

Session #2

• DL framework and scientific computing package
• tensor nd array
• numpy goto package for nd array
• Interoperable with numpy
• Torch based on Lua Programming Language
• Now maintained by Facebook

Deep Learning Features of Pytorch

• Pytorch packages
• torch - Top level package
• torch.nn - Build neural networks
• torch.autograd - Differentiable tensor operations - Derivative Calculations
• torch.nn.functional - loss, activation and convolution functions
• torch.optim - Optimization operations like SGD and Adam
• torchvision - image transformations of computer vision
• torch.utils - Dealing with Datasets
• Preferred framework for research
• For computing derivatives, computation graph
• pytorch uses dynamic computation graph

Session #3

• Anaconda python package manager
• conda install Pytorch -c Pytorch
• pip install torchvision
• cuda 9.0 is stable
• install pytorch and torchvision
• conda list pytorch
• Data + Knowledge + Architecture = Enormous Software

Session #4

• GPU - Graphics Processing Unit
• Good at handling Specialized Computing
• Parallel Computing
• Smaller computation carried out simultaneously
• NN are embarrassingly Parallel
• Optimize across the entire stack
• Cuda toolkit
• GPU + Libraries
• Supported nvidia gpu
• Much of Pytorch written in Python
• Move critical functions to c/c++
• pytorch supports multiple gpus
• GPU (CUDA - cuDNN) - Pytorch on top of Cuda
• Technology built on top of layers
• Paper (GPGPU Computing)

Happy Mastering DL!!!