Day #190 - Neural Ordinary Differential Equations

The video for this session is available in link. It Starts 24:16 Seconds

Key Summary

• Eulers ODE one pager (Numerical way to solve the problem) from Khan Academy basics video. How we can numerically approximate to find the tangent values

• ODE Solvers - Done with Numerical Solvers
• Eulers methods considered, Modern solvers have evolved
• Resnet and Euler are similar

• Update to every layer and add current hidden
• NN and ODE

• Dynamics between layer with below changes
• Continuously define with depth
• Training ODENet
• Approximate derivative

• Further Changes with adaptive solver
• ODENet
• Derivatives, Chain rules
• Recover Adjoint Sensitivities / Automatic differentiation 

• Resnets replaced with ODENet

• Errors are within Error Tolerance
• ODE to fit gradient descent
• For time series models

• ODE on RNN
• Defined States all times 
• Handle irregular data

Benefits for timeseries and RNN models

• For Density Modelling

Paper - Link 

Its Math heavy but at least high level we know this is another way to compute derivatives while training NN, As practitioner this is all within libraries but good to know the internals and advancements.

Next Reads 
NIPS papers 
NIPS best papers 

Happy Mastering DL!!!