Day #145 - ML Paper Learning Notes

Article - Link - Prediction, Interaction, and User Behaviour

Summary of Learning's

• Classifying user activity in Smartphone using Machine Learning Model (Random Forest)
• Generating text using CharRNN
• RNN and LSTM with Keras

Article - Continuing Creative Sequences
Code - Link
Paper #1 - Time-series Extreme Event Forecasting with Neural Networks at Uber

Key Learning

• Time-series modeling based on Long Short Term Memory (LSTM)
• Objective is to Estimate peak electricity demand, traffic jam severity and surge pricing for ride sharing
• Derived new feature variables - Weather information (e.g., precipitation, wind speed, temperature)
• Derived new feature variables - City level information (e.g., current trips, current users, local holidays)
• Features vectors are then aggregated via an ensemble technique
• Three criteria for picking a neural network model for time-series: (a) number of timeseries (b) length of time-series and (c) correlation among the time-series. If (a), (b) and (c) are high then the neural network might be the right choice

Paper #2 - AcctionNet: A Dataset Of Human Activity Recognition Using On-phone Motion Sensors
Network Architecture - The FCNN architecture

convolution2d_input_1: InputLayer
convolution2d_1: Convolution2D
activation_1: Activation
convolution2d_2: Convolution2D
activation_2: Activation
maxpooling2d_1: MaxPooling2D
dropout_1: Dropout
flatten_1: Flatten
features: Dense
activation_3: Activation
dropout_2: Dropout
dense_no_soft: Dense
activation_4: Activation

Classes, Biking, Downstairs, Stationary Biking, Jumping, Lunging, Running, Squatting, Standing, Upstairs, Walking

Human Activity Recognition Using Smartphones Data Set 

Happy Learning!!!

Day #144- Uber Engineering - NLP, Machine Learning Notes

Uber engineering blog provides great insights on ML / Big Data / NLP Initiatives and guidelines. Sharing Some of Key Learning's from blog posts.

Link #1 - Forecasting at Uber

Need for Forecasting

• Identify High Demand Areas
• Predict User Supply Demand
• Identify Trends, Seasonality, Competition, Pricing
• Data Driven marketing decisions

Visualization Key Points

• Daily Trends
• Hourly Trends
• Weekly Trends
• Weekend Trends

Forecasting Techniques

• Statistical - ARIMA, Holt-Winter
• ML Based - RNN, Quantile Regression Forest (QRF), Gradient Boosting Trees (GBM), Support Vector Regression (SVR), Gaussian process regression (GP)

Summary

• Understand Trend, External Factors like Weather, Concerts
• Quantile Regression Forest (QRF) -  provide predictions at percentiles
• Gradient Boosting Trees (GBM) - a prediction model in the form of an ensemble of weak prediction models, typically decision trees 
• Support Vector Regression (SVR)
• Gaussian process regression (GP) - The prediction is probabilistic (Gaussian)
• Central Limit Theorem - Almost all measurable "random" variables in real world follow some kind of normal distribution
• Binomial - Only two outcomes of trail
• Poisson - Events that take place over and over again. Rate of Event denoted by lambda

Link #2 - COTA: Improving Uber Customer Care with NLP & Machine Learning

NLP Models built to interpret

• Phonology
• Morphology
• Grammar
• Syntax
• Semantics

Character level, word-level, pharase-level or sentence level or document level language modelling

Key Concepts in Implementation

• Fetch Trip Data
• Fetch Ticket Text
• Preprocessing (Stemming, Lowercasing, Stop-word removal, Lemmatization)
• Feature Engineering (LSI / TF_IDF)
• Cosine Similarity
• ML Algorithm for Ranking (Predictions - Issues / Solutions)
• Topic modeling transformation is carried out on the bag-of-word representation

Link #3 - Applying Customer Feedback: How NLP & Deep Learning Improve Uber’s Maps

Key Concepts

• Tickets encoded using One Hot Encoder
• Classification for type of ticket
• Alternatively used word2vec based representation
• WordCNN and LSTM based models

More Reads - Uber’s Big Data Platform: 100+ Petabytes with Minute Latency
Databook: Turning Big Data into Knowledge with Metadata at Uber

Happy Learning!!!

Day #143 - Chatbot Reading and Implementation Notes

Reading #1 - Link1
Key Point #1 - Closed Domain Based Chatbot

Key Point #1 - NLP Based Intent / Entity / Response System

• Intent
• Entity
• Response / Action

Key Point #2 - Knowledge Based Question and Answering

• Convert Query into Database Query
• User Query to Structured Database Query

Paper #1 - A Survey of Available Corpora for Building
Data-Driven Dialogue Systems

Datasets - The Second Dialog State Tracking Challenge - DSTC1 (Williams et al., 2013)
The ATIS Spoken Language Systems Pilot Corpus

Summary of Key Points

• For Dialogue based systems Speech Recognition, State Tracker, Response Tracker, Speech Synthesizer are different components
• Goal Driven Dialogue Systems - ML to identify intention / heavily hand crafted rules
• Learning Dialogue System Components - Probabilistic Models, Discriminative tasks, Using Supervised learning
• Dialogue State Tracker - K Dialogue state based tracker
• Ranking based systems, Encoding word by word, P(Action / Dialogue Tracker)
• Different corpus information and details

Paper #2 - MACHINE LEARNING FOR DIALOG STATE TRACKING

• Discriminative methods exploiting ML to implement Sequence to Sequence models
• Estimate Likelihood, Prior probability
• n-gram based/ word based Dialogue State Tracker
• Rule Based / RNN based

Paper #3 - Smart Reply: Automated Response Suggestion for Email 

• LSTM Neural network to process and predict likely response
• Semi-Supervised learning approach
• Feed forward network suggests whether or not to suggest
• Semantic Intent Clustering (Partition responses into semantic clusters)
• Key tasks - Tokenization / Normalization / Salutation removal / Remove infrequent words (Personal names, Phone numbers)

Paper #4 - AliMe Chat: A Sequence to Sequence and Rerank based Chatbot Engine

• Identify Intent
• Check Knowledge Base
• Answer Generation
• Attentive Seq2Seq model

Paper #5 - Evaluating Natural Language Understanding Services for Conversational Question Answering Systems

• Request text analysis
• Determine Contextual Information
• Response Message Generation from Knowledge Base

Chatbot Product Ideas

• Based on the person, Gender age assign appropriate Style, Tone, Personality
• Localized chatbot implementation
• Customized Response Generation based on Knowledge base and contextual information
• Leverage Data Store (Interaction History) / Knowledge Base (Content Management System)
• SMS-based chatbot

Technical Ideas

• Named Entity Recognition
• Sentiment Analysis
• Part of Speech Tagging
• Dependency Parsing

Projects - Deep Learning for Chatbots, Part 2 – Implementing a Retrieval-Based Model in Tensorflow, chatbot-retrieval

Happy Learning!!!

Data & Data Science Posters

Image Source - Quora 

Happy Learning!!!

Day #142 - CMU Database Systems - 09 Index Concurrency Control

Concurrency Control

• Method to allow concurrent operation of the shared object 
• Ensures Logical and Physical Correctness

Physical Correctness
Latch Modes
Latch Crabbing
Leaf Scans
Delayed Parent Updates

Locks vs Latches
Locks

• Protects the index's logical contents
• Held for transaction duration
• Need to be able to rollback changes

Latches

• Protect the critical sections of index's internal data
• Held for operation duration
• Do not need to be able to rollback changes (Atomic operation)

Locks

• Separates user transactions
• Protects Database contents
• During Entire transaction
• Lock Modes are shared, exclusive, update and intent
• Deadlock by waits-for, timeout, aborts
• Kept in Lock Manager

Latches

• Separate Threads
• Protects in memory data structures
• During Critical sections
• Red / Write Modes
• Kept in the protected data structure
• Multiple threads allowed to read but only one thread to write

B+ Tree Concurrency Control

• Protect threads trying to modify node concurrently
• Latch Coupling / Latch Crabbing - Allow both threads to access / modify at the same time
• Latch is in FIFO order
• Instead of locking Root and creating contention point, Alternate option Optimistic algorithm
• Assumption leaf node will be safe
• Read / write latches compatibility decides how they manage without deadlocks
• Thread that kill itself if prolonged wait / deadlock / starving
• Retry is better than waiting without timeout / Spin for milli second then restart
• Making B+ Tree threadsafe

Highlevel techniques (Acquire latches in same order, Killing and restarting)

• Page Layout
• Data Structure
• STL Iterator
• Latch Grabbing

Happy Learning!!!

Day #141 - New Environment Video - Audio - Text

Happy Learning!!!

Day #140 - Research Areas - Data Science

Four Key Areas

• Bayesian Deep Learning 
• Hierarchical reasoning models
• Unsupervised image understanding
• Computer vision (K-shot learning) 

Drawbacks of Deep Learning

• Uninterpretable black boxes
• Easily Fooled, what a model not know
• Crucially relies on Big Data

Drop out

• Randomly setting network units to zero
• Cited hundreds of times
• Improves performance reduces overfitting

Bayesian Deep Learning 

• Connect Deep Learning and Bayesian Probability Theory
• Bayesian neural network to replace dropouts
• Place prior p(W) distribution of weights
• Given Dataset X,Y compute posterior p(W/X,Y)

Simplistic Reference

• In practical terms given point x:
• drop units at test time
• repeat 10 times
• and look at mean and sample variance.

y = []

for _ in xrange(10):

y.append(model.output(x, dropout=True))

y_mean = numpy.mean(y)

y_var = numpy.var(y)

Bayesian deep learning models typically form uncertainty estimates by either placing distributions over model weights, or by learning a direct mapping to probabilistic outputs.

Reference

https://www.cs.ox.ac.uk/people/yarin.gal/website/PDFs/2017_OReilly_talk.pdf

https://alexgkendall.com/computer_vision/bayesian_deep_learning_for_safe_ai/

https://github.com/sjchoi86/bayes-nn

Computer vision (K-shot learning) 

Paper #1 - Efficient K-Shot Learning with Regularized Deep Networks

Keypoints

• Learning from very few training
• samples or k-shot learning, is an important learning paradigm that is widely believed to be how humans learn new concepts

Related Work

• Automatically learn feature representations where objects of the same class are closer together
• LSTM-based meta-learner that uses its state to represent the learning updates of the parameters of a classifier for k-shot learning

Paper Contribution

• Grouping neuron by activations for layer-wise clustering of parameters 
• Hybrid loss function for k-shot learning consisting of cross-entropy loss as well as triplet loss among the k-shot data
• Reinforcement learning based mechanism to efficiently search for the optimal clustering of the parameters

Next Post will look into below topics

• Hierarchical reasoning models
• Unsupervised image understanding

Happy Learning!!!

Paper Analysis - Driver Behavior and Interaction with Automation

Paper - https://arxiv.org/pdf/1711.06976.pdf

Key Objectives Studied and Techniques Identified for those key features

• Capture Driver State (Fine-grained face recognition, facial expression recognition, detect emotional stress)
• Driver body position (Driver Pose, Driving Behavior, Driver Alertness monitoring)
• Driver Scene Perception (Places / Landscapes
• Vehicle State - Driving Behavior, Steering Commands

Frame Details

• Three or four Logitech C920 webcams record at a resolution of 1280x720 at 30 frames per second within the car

Software

• Cam2hd is a program written in C that opens and records all camera data
• Lighthouse is a python script that sends information about each trip to Homebase. Information sent includes timing information for the trip, GPS data, power consumption, temperature, and available external drive space
• RIDER Database - A PostgreSQL database is used to store all incoming trip information, as well as to house information about all trips offloaded to a storage server

A Good start to understand/reuse this approach/hardware recommendation for large-scale video analytics projects

Happy Learning!!!

Tesla Autopilot Object Detection

Came across this interesting article How a Tesla computer “sees” the streets of Paris



Labels in Frame

• Lanes (Yellow)
• Speed (White)
• Type of Vehicle (Red)
• Signals in that Lane
• Alerting when vehicle before is stopped
• Dimension (Length of Vehicle)
• Pedestrian Detection / Speed
• Boundary of detected objects

Even if we take a frame and split into candidate windows, The operation is near real-time classification. It would be interesting to know the technique here RegionalCNN / CNN / Mask-RCNN how this network would like. Since this runs on the vehicle model may be deployed within the vehicle. The model size/frame size / overall architecture would be interesting to learn.

One more interesting Night Vision of Tesla I spotted recently

To what extent are you willing to trust self-driving cars with your life? 🚗🤖#SelfDrivingCars #cars #Tesla #IoT #ElonMusk #innovation #technology #ArtificialIntelligence #AI #Industry40 pic.twitter.com/oCv0uh1366

— The Futurist 🚀 (@TheFuturist007) March 10, 2019

Tesla Model 3 saved me from r/teslamotors

ADAS - Link

HT: @MikeQuindazzi
copy @Antgrasso @Fisher85m

An #OpenSource software ecosystem for #AutonomousVehicles >>> @CBinsights via @MikeQuindazzi >>> #AI #IoT #mpgvip #MachineLearning #ComputerVision #IIoT #4IR >>> https://t.co/VipVBP4zO8 pic.twitter.com/c6Thdf2cNC

— The Cyber Spanish (@cyber_spanish) May 29, 2020

🚨🚨🚨
Tesla Model 3 plows info overturned truck on highway. I’m sure the driver was paying complete attention to the road and wasn’t relying on autopilot because he was told the car could drive itself....$TSLAQ pic.twitter.com/cHjueqH0j4

— Fred Lambert is never getting his Roadster 🐓 (@jsin86524368) June 1, 2020

Software Engineering Within SpaceX

Tesla is coming up with driver monitoring features. From the link the 

Detection / Classification Classes are

• BLINDED
• DARK
• EYES_CLOSED
• EYES_DOWN
• EYES_NOMINAL
• EYES_UP
• HEAD_DOWN
• HEAD_TRUNC
• LOOKING_LEFT
• LOOKING_RIGHT
• PHONE_USE
• SUNGLASSES_EYES_LIKELY_NOMINAL
• SUNGLASSES_LIKELY_EYES_DOWN

Mercedez enters the Race

Ref - Link

Happy Object Detection and Driving!!!

The Value of Data

The value of data across the chain as we look at data from different perspectives.

Early Stages of Data Analysis

• Data Collection
• Data Reporting
• Dashboard Reporting

Mid Level Data Analysis

• Metrics / Measures for Creation (Business Intelligence / Trends)
• Predictive (Forecasting)

Advanced Level

• Machine Learning Based Models
• End to End Automated Workflow chain with all levels of Data Analysis

Pain Points of BI

• Reactive Mode - It doesn't help us to know the future but rather learn from past
• Data Distribution - Cannot fit Data and Distributions to understand better, Data can be fit into any form of pattern / distribution. BI does not help us Deep Dive to identify that
• Tight Schema Bound Architecture - Models, Dimensions are Static, Dynamically cannot change the facts, a measure without changing underlying ETL

Happy Understanding with Data!!!

Day #139 - Video Analytics Case Study - Retail Scene Gun Detect Alert Situation

Summary of key points from papers studied, Sample projects, Code base to follow it further. This is a summary and bookmark. Credits to bookmarked papers / authors

Papers Studied
Paper #1 - Real Time Monitoring of CCTV Camera Images Using Object Detectors and Scene Classification for Retail and Surveillance Applications

Key Points

• AlexNet has 60 million parameters and 650,000 neurons, consists of five convolutional layers. Those layers are followed by max-pooling layers, and three globally-connected layers with a final 1000-way softmax layer

CNN Architecture Overview

• Kernels are learnable filters
• Pooling layer (sub-sampling) reduces the dimensionality of feature map
• Output of Softmax function categorical distribution

Datasets

• Knives Images Database
• Internet Movie Firearms Database
• EgoHands
• ImageNet
• Pistol-Detection-in-Videos
• Knife Image Database 

Application Architecture
Input CCTV Video -> Detection using CNN -> Push Notification to Mobile

Paper #2 - Crime Scene Prediction by Detecting Threatening Objects Using Convolutional Neural Network

From the scene the objects are extracted, CNN is used to identify. Detecting weapons is a bigger challenge than identifying / classifying it.

Paper #3 - Automatic Handgun Detection Alarm in Videos Using Deep Learning

• Reformulated problem to reducing number of False Positives

Detection Models

• Sliding Window Approach
• Large number of candidate windows
• Runs Classifier on all windows
• HOG based model
• Good at 0.07 frames per second / pedestrian

Region Proposals

• Region based CNN
• Approach selects accurate candidate regions
• CNN to extract features, SVM to classify them
• Good at 7 frames per Second
• Alarm Activation Time per Interval (AATI)

Paper #4 - Automated Detection of Firearms and Knives in a CCTV Image (Michał Grega*, Andrzej Matiolanski, Piotr Guzik and Mikołaj Leszczuk)

Challenges

• Blurryness
• Low Resolution

Knife Detection Algorithm

• Input Image -> Sliding Window -> Feature Extraction -> SVM Classification -> Decision

Firearm Detection Algorithm
OpenCV Techniques used

• Background Detection
• Canny Edge Detection
• PCA

Neural Network operates on the Image Obtained after PCA (Reduced Dimensions)
OpenCV Post Evaluation

• Dialation
• Erosion
• Difference between two images

Experimentation - Tested this haar cascade based implementation, Works fine for Gun Detection
Results

Other Interesting Projects
Real Time Implementation of Gunshot Detection System
tensorflow-gun-detection
Deep Neural Net Approach To Identify Guns

Happy Learning!!!

Video Analytics Use Cases

Real-world Use Cases from Surveillance / Security Cameras (Simple to Medium)

• Object Detection
• Face Detection
• Slip and Fall
• Loitering
• Crowd Detection
• Abandoned Objects
• People Counting
• Gender-Based People Counting
• Attendance / Visitor Tracking
• Traffic Signal / Lane Detection / License Number Tracking
• Emotion Analytics
• Parking Analysis
• Mob / Crowd / Weapon Detection in Public Places

Advanced Video Analytics Use Cases (Complex) - Deep Learning Use Cases

• Medical Processing X-Ray / Genome Data 
• Tumor Identification
• Autopilot Self Driving
• Smart Farming - Fruit Ripening / Fruit Classification
• OCR Recognition / Financial Data Analysis
• Smart Home management / Intrusion Detection
• Online Media Moderation - Abuse / Content Rating / Detection

Manual Fruit Classification

Computer Vision Size Classification

Packing Validation




Grape Detection and Segmentation



Smart City Analytics - ML + DL

• Parking Analytics
• People Counting
• Face Recognition
• Intrusion Detection
• Device Monitoring And Alerting

Happy Learning!!!

Day #138 - Opencv Based Blob Detection and Image Extraction

Happy Learning!!!

Day #137 - RBM VAE

Happy Learning!!!

Day #136 - RNN Session

Notes from session - Andrej Karpathy, Research Scientist, OpenAI - RE•WORK Deep Learning Summit 2016 

RNN

• Operate on Sequences
• Image Captioning (Produce Sequence of words)
• Sentiment Classification / Machine Translation
• Video Classification
• RNN has a state which update everytime when vector comes in x
• RNN can process vectors so it is encoded
• Use RNN to generate sequences after learning from dataset
• Interesting examples of RNN code, function, bogus comments, syntactically very few mistakes
• min-char-rnn.py, char-rnn projects to look out for (https://gist.github.com/karpathy/d4dee566867f8291f086)
• Detect and describe objects in image in single pass

Happy Learning!!!

Deep Dive PySpark Examples - Big Data Setup - Part II

After experimenting a bit of pyspark I feel Its much better to handle with R / Python. Most of things we can achieve are repetitive between R /Python / Spark / SQL.

• Data Pipeline tasks at DB Level
• One Hot Encoding also can done with basic TSQL Code
• While working in NLP it makes sense to use TF-IDF Vectorizers

Happy Learning!!!

Day #135 - Research paper - Human-Centered Autonomous Vehicle Systems

Its a long day and only now I got time to post on this interesting paper - Human-Centered Autonomous Vehicle Systems

I love this paper for the practicality and applicability of this kind of systems. Autonomous Systems and Humans are dependent on each other to learn / teach and understand each other for Safer modes of transportation. In such situations systems have to share common responsibility and dependency to make the rides safer.

This paper underlines this important idea and the shared responsibility across multiple levels. Supervised learning, Personalization, better communication for Safer Navigation

Some key lines (copied from paper)

• Machine learning is primarily used for the scene understanding problem but not for any other aspect of the stack driving scene
• Perception, motion planning, driver sensing, speech recognition, and speech synthesis are all neural network models
• Calibrate distraction, fatigue, cognitive load, emotional state, and activity (uses facial landmark configuration and facial motion analysis)
• Communicate the degree of uncertainty in the neural network prediction, segmentation, or estimation about the state of driving scene
• Arguing machines framework (detailed in [12]) to provide human supervision over the primary perception-control system

Nice Lines
"both humans and AI systems have flaws, and only when the share autonomy paradigm is considered at the system level do those flaws have a chance to be leveraged to become strengths"

The same principles should apply for chatbots for better communication. 

Happy Learning!!!

Predictions by 2025

A quick write up on my predictions for technology/education and other sectors by 2025. Will revisit it again after a few years.

• Large scale deployment of Video Analytics in Scale for Home monitoring / Attendance Monitoring / Retail / Gesture-based tracking. The scale of adoption (30~40% in Indian Middle-Class Home market)
• Smart Cities Reality Check - Bringing together IoT + Analytics + Solar Energy into better / optimum utilization of resources (In metros compliance would be there for all SEZ, Commercial establishments - 50-60%)
• Redefined Education - Customized Learning based on Individual patterns / Choices  (30~40% in Indian Middle-Class Education Market, Affordable, Free and Customizable)
• Healthcare Analytics - More Adoption for Automated / Proactive alerting from Video / Data Analytics (30% of diagnosis would be done by Analytics- prediction)
• More Open Source Quality Education Tools - More new you-tube similar channels targeting education domain / free and open-source quality education for Higher studies (Udemy, Coursera, Khanacademy probably focused on Engineering, Medical, Arts, Science very specialized channels with focused groups across the globe)
• Drone / Quad-copters / Air Ambulance - Based on Solar + Drone Technology (They will have a decent market share and adoption for delivery / carry the minimum load)
• Battery / Electric cars vs Fuel Based - This would still be the same unless the cost of product / better innovation takes place to reduce the cost of manufacturing for Battery cars (20~30% Battery cars in Indian Automotive market)
• Big Data + Analytics - More the data, More the chaos, Real-time meaningful analytics from huge data sources vs computing power in Home devices or Smart Homes will still be a challenge 

#RandomThought

Update May 29/ 2020
Corona crisis has updated a lot of things. Glad to see many startups built on unreasonable practices are gone from the market.

COVID Accelerated the below things.

• Online Education and free education is becoming a reality. 
• Work from Anywhere is becoming a reality.
• Every Retail store is becoming a Digital Store
• AI in healthcare / screening 

Link
Key Notes

• Device to custom food with nutrients
• Autonomous vehicles market share increase by 50%
• Electric vehicles market share increase by 50%
• Use of Green energy
• Other relevant 2025 predictions listed above
• SpaceX Mars Colonization
• Global Warming
• Impact of China on the World

THE WORLD IN 2040

Interesting Read world in 2040

You are not born educated,
You are not born learned,
You are not born experienced,
You are not born matured.

Education trigger learning,
Learning trigger experience,
Experience trigger maturity.

Maturity becomes your habit & foundation.

— LearnLifeWealthTravel (@AnyBodyCanFly) July 4, 2020

Happy Learning!!!

Day #134 - Summary of learning's from stitchfix

Algorithms Summary from stitchfix Blog

• Collaborative filtering algorithms  for recommendations (User - User / Item - Item Recommendations)
• cosine similarity, Finding Image based Similarity 
• NLP on client feedback to create score of feedback (Possibly positive / Negative / Neutral based on words used)
• Travelling Salesman problem to identify optimal delivery cycle
• Keep track of every touch point of transaction to predict / pro-active on next move
• Markov chain model - Demand Modelling, This could be classic machine learning model that includes several feature variables (Seasonality / Trend / Style etc..)

Notable Quotes

• Engineers Shouldn't Write ETL
• Data Platform team enables data scientists to carry algorithm development all the way from concept to production

Happy Learning!!!