Day #140 - Research Areas - Data Science

Four Key Areas

• Bayesian Deep Learning 
• Hierarchical reasoning models
• Unsupervised image understanding
• Computer vision (K-shot learning) 

Drawbacks of Deep Learning

• Uninterpretable black boxes
• Easily Fooled, what a model not know
• Crucially relies on Big Data

Drop out

• Randomly setting network units to zero
• Cited hundreds of times
• Improves performance reduces overfitting

Bayesian Deep Learning 

• Connect Deep Learning and Bayesian Probability Theory
• Bayesian neural network to replace dropouts
• Place prior p(W) distribution of weights
• Given Dataset X,Y compute posterior p(W/X,Y)

Simplistic Reference

• In practical terms given point x:
• drop units at test time
• repeat 10 times
• and look at mean and sample variance.

y = []

for _ in xrange(10):

y.append(model.output(x, dropout=True))

y_mean = numpy.mean(y)

y_var = numpy.var(y)

Bayesian deep learning models typically form uncertainty estimates by either placing distributions over model weights, or by learning a direct mapping to probabilistic outputs.

Reference

https://www.cs.ox.ac.uk/people/yarin.gal/website/PDFs/2017_OReilly_talk.pdf

https://alexgkendall.com/computer_vision/bayesian_deep_learning_for_safe_ai/

https://github.com/sjchoi86/bayes-nn

Computer vision (K-shot learning) 

Paper #1 - Efficient K-Shot Learning with Regularized Deep Networks

Keypoints

• Learning from very few training
• samples or k-shot learning, is an important learning paradigm that is widely believed to be how humans learn new concepts

Related Work

• Automatically learn feature representations where objects of the same class are closer together
• LSTM-based meta-learner that uses its state to represent the learning updates of the parameters of a classifier for k-shot learning

Paper Contribution

• Grouping neuron by activations for layer-wise clustering of parameters 
• Hybrid loss function for k-shot learning consisting of cross-entropy loss as well as triplet loss among the k-shot data
• Reinforcement learning based mechanism to efficiently search for the optimal clustering of the parameters

Next Post will look into below topics

• Hierarchical reasoning models
• Unsupervised image understanding

Happy Learning!!!