Day #334 - Exploring - Featuretools

Have been listening/hearing about Feature generation, feature management. There are a couple of tools/frameworks in this perspective.

For a typical ML product level use case

• Who defines the problem - Domain Expert / Product Manager
• Who knows the data sources - BA / Database Developer / Product Manager
• Raw Data -> Processed data - DB Developer
• Data Exploration / Analysis / Feature Creation - BI / DB / ML Developer
• Model Development / Validation - ML Developer
• Deployment / Monitoring / Improvement - Devops / ML Developer

Feature store handle the part between raw data - data aggregation - feature generation/feature engineering

Installing Featuretools

Analysis - Basically like connecting few tables, doing that analysis of unique, average its all taken care after you define the entities, It is like prebuilt analysis based on identified associations

Experimenting this on colab - Colab notebook link 

From Link , Feature comparison between different feature stores

Paper - Link

Key Notes

• Handling Data Ingestion 
• Aggregating data from diverse sources
• Access controlled and versioned

Key Offerings

• Automated Feature Generation
• Access to generated feature
• Data Privacy / Data Governance
• Data Visualization

My Thoughts

• Today with all cloud trend all the data OLTP, OLAP, SQL, NoSQL sit next to each other 
• Generating reports aggregating all sources in near real-time fashion is possible
• Some features/variables can be pulled from OLTP tables
• In a Data Lake / DW, Some of the insights would be already present in computed reports
• Metadata management would already be available in the system which will handle data quality aspects
• ML systems will work together as part of larger Data ecosystem comprising of OLTP, OLAP, SQL, NoSQL system. A lot of feature store workloads are already handled by other pieces. 

More Reads

• Featurestore
• Hopsworks - FS
• FSS
• Rethinking - FS
• Featuretools
• FeatureEngineering
• FS Guide
• FS Detailed Guide

Keep Thinking!!

Tracking and Pedestrain Paper Reads

Paper #1 - Object Tracking for Autonomous Driving Systems
keynotes

• The Different components of Autonomous driving System

I'm keen on Tracking - Prediction - Planning aspects
MOT - Multiple Object Tracking

• Record and track moving objects over time
• In Autonomous System tracking cars, pedestrians, signals, passing vehicles everything is key

Deep Learning in Video Multi-Object Tracking

• Detection - Object Detection, Bounding boxes
• Feature Extraction - CNN or Siamese networks employed
• Affinity - Cosine Similarity, Euclidean distance, Intersection over union
• Association - Matching trackets

Real-world challenges

• Camera Motion
• Occlusions

Deep Sort - DeepSORT extends SORT by incorporating appearance information for each tracked object into the affinity stage. The underlying Kalman filter and target state definition are unchanged, but a convolutional feature extraction model is used to process bounding box crops of targets into feature vectors

Novelty
Distractor-Aware Siamese Region Proposal - Networks - DaSiamRPN is a single object tracker that combines a Siamese feature extraction network with a novel training strategy to learn distractor-aware features.

Paper #2  - Estimating Pedestrian Densities, Wait Times, and Flows with Wi-Fi and Bluetooth Sensors

Key Notes
Tracking Approaches

• Infrared, Thermal Sensors
• Bluetooth, Mobile Phones
• Automated Pedestrian Counting Technologies

Pedestrian Counting Techniques

• Anonymize, filter and aggregate traces of pedestrians
• Wifi, MAC-based approach

Paper #3 - Autonomous Vehicles that Interact with Pedestrians: A Survey of Theory and Practice
Key Notes
Different pedestrian vs Environment factors are

Decision-making Factors

SIMPLE ONLINE AND REALTIME TRACKING WITH A DEEP ASSOCIATION METRIC
Key Notes

• Simple online and realtime tracking (SORT)
• Standard Kalman filter with constant velocity motion and linear observation model
• Kalman filtering, also known as linear quadratic estimation (LQE)
• By using simple nearest neighbor queries without additional metric learning

SORT for Tracking - Git Project

Good Reads
ML6 Internship: Pedestrian tracking over multiple non-overlapping camera viewpoints

Keep Thinking!!!

Weekend Reads - Fashion Virtual Dress Creation - Papers

Paper #1 - M2E-Try On Net: Fashion from Model to Everyone
Key Notes

• Pose alignment network (PAN)
• Texture refinement network (TRN)
• Fitting network (FTN)

Pose alignment network - (PAN) to align the model and clothes pose to the target pose. Each dense pose prediction has a partition of 24 parts

Texture refinement network (TRN) to enrich the textures and logo patterns to the desired clothes
Texture details, region of texture, binary mask with the same size. Merged images while still preserving the textual details on the garments

Fitting network (FTN) to merge the transferred garments to the target person images. Generative network to generate fashion images from textual inputs

Fitting Network is an encoder-decoder network, including three convolution layers as the encoder, six residual blocks for feature learning, followed by two deconvolution layers and one convolution layer as the decoder

Code - https://github.com/shionhonda/viton-gan

Paper #2 - VITON-GAN: Virtual Try-on Image Generator Trained with Adversarial Loss
Key Notes

• U-net generator and thin plate spline (TPS)
• Human parser
• Pose estimator

GANs are able to generate fine, high-resolution, and realistic images because adversarial loss can incorporate perceptual features that are difficult to define mathematically

Try-on module (TOM)

• Trained adversarially against the discriminator that uses the TOM result image
• Person representation as inputs and judges whether the result is real or fake VITON-GAN generated hands and arms

Paper #3 - The Conditional Analogy GAN: Swapping Fashion Articles on People Images

Key Notes

• GAN - a model G that learns a data distribution from example data, and a discriminator D that attempts to distinguish generated from training data
• These models learn loss functions that adapt to the data, and this makes them perfectly suitable for image-to-image translation tasks

• (cGAN) learns to generate images as function of conditioning information from a dataset, instead of random noise from a prior, as in standard GAN

Training of the CAGAN model involves learning a generator G to generate plausible images which fool a discriminator D. The discriminator D needs to answer two questions:

• does an image x look reasonable, i.e. indistinguishable from the training distribution of human images {xi}?
• does the article y look well-painted on the human model image x

Neural Re-Rendering of Humans from a Single Image

More Reads

GENERATIVE ADVERSARIAL NETWORK-BASED VIRTUAL TRY-ON WITH CLOTHING REGION

Keep Thinking!!!

Research Paper Reads - People Counting

Paper #1 - PaDNet: Pan-Density Crowd Counting

Key Notes

• Density-Aware Network (DAN) contains multiple subnetworks pretrained on scenarios with different densities
• Capturing pandensity information
• Feature Enhancement Layer (FEL) effectively captures the global and local contextual features
• Feature Fusion Network (FFN) embeds spatial context and fuses these density-specific features

Real World Challenges

• Inconsistent densities due to camera perspective

Existing Methods

• Sliding window detector
• Regression-based approaches
• Hand-crafted features
• detection-based methods affected by severe occlusions

Switch-CNN

• Switch-CNN through training the switch classifier to select the optimal regressor for one input patch
• Each subnetwork of Switch-CNN is trained on a specific density subdataset and thus cannot utilize the whole dataset
• High computation complexity in predicting the global and local contexts

Network Design

• FEN extracts low-level feature of image
• DAN employs multiple subnetworks to recognize different density levels in crowds and to generate the feature map

Regression Based Methods

• Mapping from low-level features extracted from local image patches to the count

Features

• Extracted features include foreground features, edge features, textures, and gradient features such as local binary pattern (LBP), and histogram oriented gradients (HOG)
• Regression approaches include linear regression [24], piecewise linear regression [25], ridge regression [26], and Gaussian process regression

MCNN architecture

• Five-branch contextual pyramid CNN
• GANs-based method to generate highquality density maps

Density-Aware Network (DAN)

• Input: input crowd image patches dataset S
• Output: output the parameters ΘPaDNet
• Init: Dividing the whole image patches S into N clusters S1, S2...SN via K-means clustering algorithm.
• P is the number of people in an image patch, dij represents the distance between the ith subject and its jth nearest neighbor

Paper #2 - Point in, Box out: Beyond Counting Persons in Crowds
Crowd Counting Challenges
Heavy occlusions, perspective distortions, scale variations and varying density of people
Combine the detection result with regression result for crowd counting

Implementation
Point-level annotations on person heads

Novelty

• Online pseudo ground truth updating scheme which initializes the pseudo ground truth bounding boxes from point-level annotations
• Novel locally-constrained regression loss
• Curriculum learning strategy

Key Lessons

• Patch-based density estimation

Paper #3 - Revisiting Perspective Information for Efficient Crowd Counting

• Estimate crowd counts via the detection of each individual pedestrian Regression of density maps 
• Crowd counting is casted as estimating a continuous density function

Multi-scale architecture of convolutional neural networks (CNNs) to regress the density maps at different resolutions
Average distance from certain head at j to its K-nearest neighbors (K-NN)

Datasets

• ShanghaiTech
• WorldExpo’10
• UCF CC 50
• UCSD

Paper #4 - CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly
Congested Scenes

• Convolutional neural network
• Front-end for 2D feature extraction and a dilated CNN for the back-end
• Multi-column based architecture (MCNN) for crowd counting.
• Dilated convolutional layers have been demonstrated in segmentation tasks with significant improvement of accuracy
• Dilated convolution shows distinct advantages compared to the scheme of using convolution + pooling + deconvolution. 

Deep Learning-Based Crowd Scene Analysis Survey

Detection based approaches

Datasets

Make it a High / Medium / Low crowd scenario

Code
Congested Scene Recognition called CSRNet 
Ref Code

Dilated convolutional layers to aggregate the multiscale contextual information in the congested scenes.

More Papers
Benchmark data and method for real-time people counting in cluttered scenes using depth sensors
Learning Spatial Awareness to Improve Crowd Counting
Image Crowd Counting Using Convolutional Neural Network and Markov Random Field
Fast Video Crowd Counting with a Temporal Aware Network
Paper with Code - Reference
Example - Keras - Dilated Convolution
7 Technologies that Count People (Buildings & Offices)

Keep Thinking!!!

Research Paper Reads - Edge Analytics

Microsoft has made a lot of progress on edge analytics. Sighted few papers which provide good pointers on case studies, implementation examples.

Paper #1 - Demo: Video Analytics - Killer App for Edge Computing
Key Notes
Microsoft Video Analytics Stack

• Combination of Edge + Cloud Analytics
• Lite and Heavier version of models
• Hybrid approach

Pipeline Steps

• Decoder
• Background subtractor
• Deep neural network

Performance Challenges

• Edge device is not overloaded
• Edge device used for cheap filtering
• Data sent between the edge and the cloud does not overload the network

Cloud Features

• Efficient cross-camera analytics
• Querying stored videos

Paper #2 - Traffic Video Analytics 
Key Notes

• Ground Truth - under-counts, over-counts
• Ground truths were obtained using extensive labeling
• Line-counter is that it captures the “state transition” of the lines
• State of the line changes from unoccupied to occupied, and then back to unoccupied, before it increases the count for said line

Paper #3 - The Emerging Landscape of Edge-Computing.
Key Notes

• Edge Computing - tolerate cloud outages and the scarcity of network bandwidth
• This slide on real-world deployments is very impressive

• Existing edge applications often consist of an ensemble of custom and off-the-shelf containers
• Use Case - Cracks can cause the wheel to break and derail the entire train
• Oil - ExxonMobile continuously monitors its multi-million dollar oil rigs
• Network bandwidth and reliability drive the use of edge computing
• Autonomous Edge devices

Visor: Privacy-Preserving Video Analytics as a Cloud Service

A Review on Edge Analytics: Issues, Challenges, Opportunities, Promises, Future

Directions, and Applications

Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications

Deep Learning and Machine Vision for Food Processing: A Survey

Happy Learning!!!

Automated Stores

Store #1 - Automated Stores - nanostore

• Identification - Credit Cards
• Products
• Weight Sensors
• Location

Key Components

• Android Device for restocking
• AiFi Smart Sensing
• AiFi OASIS Auto-Checkout System

Link

Project #1 - Link

High-Level Architecture

Project #2 - Link

Next Store Solutions

Key Notes

• QR code to Register
• Self-checkout
• Wechat payment

Alibaba Retail Store Innovation

• Innovation in Fresh Foods
• Instore digital support tools
• Product online reviews / Recommendations
• Cooking instore support
• Self-checkout counters
• Conveyor belts
• Order online pick up offline
• 30mins for 3Kms radius

Store = Fulfilment centre + Market + Food Court + Warehouse

Smart Cart


Smart Cart

• Self-checkout on wheels
• Built-in scanner
• Scan & Place product in the cart
• Real-time tally of bill and tax
• Weight sensor-equipped
• Shopping recommendations online

Smart Cart #2

• Auto checkout are
• Drop goods
• Pay and pick

Automated Billing and Robotic Delivery

• Trolley with items sent
• Items scanned
• Robots with packed items navigate and deliver in lobby

Happy Learning!!!

A Survey of Deep Learning Techniques for Autonomous Driving

A Survey of Deep Learning Techniques for Autonomous Driving
Key Summary

• Data Sources - Cameras, radars, LiDARs, ultrasonic sensors, GPS units and/or inertial sensors

AI for different components

• Perception and Localization - Segmentation
• High-Level Path Planning  - Road Ahead / Turns 
• Behavior Arbitration, or low-level path planning - Steering
• Motion Controllers

Recurrent Neural Networks (RNN) are especially good in processing temporal sequence data, such as text, or video streams
Long Short-Term Memory (LSTM) [17] networks are non-linear function approximators for estimating temporal dependencies in sequence data
Reinforcement Learning using Partially Observable Markov Decision Process (POMDP)
formalism

• Perception and Localization - Segmentation
• Object detection and recognition, semantic segmentation
• Tesla® tries to leverage on its camera systems, whereas Waymo’s driving technology relies more on Lidar sensors4
• Waymo - 5 Radars, 8 Camera's
• Tesla - 8 Cameras, 12 ultrasonic sensors, one forward-facing radar

Driving Scene Understanding

• Single-stage detectors do not provide the same performances as double stage detectors but are significantly faster.
• The main disadvantage of using a LiDAR in the sensory suite of a self-driving car is primarily its cost

Semantic and Instance Segmentation

• Drivable area, pedestrians, traffic participants, buildings
• SegNet, AdapNet, Mask R-CNN

Localization

• Localization algorithms aim at calculating the pose (position and orientation) 
• The structure of the environment can be mapped incrementally with the computation of the camera pose - Simultaneous Localization and Mapping (SLAM)

Happy Learning!!!

Webinar - Keynotes - From Notebook to Kubeflow Pipelines

Key Notes

• Kubeflow - Deployment of ML workflow on Kubernetes

• Different companies different platforms, Leverage Kubeflow as a deployment platform

• ML Aspects of Deployment

• CI / CD requirements, Iterate and push code to production

• Complexities in ML, Diverse tools Notebook / IDE, Notebook to pipeline, CUJ - Critical user journey

• Kubeflow pipelines. Data versioning, snapshots, Tools - Kale, Arrikto (Data Management)

• Deploying Challenges, Composable pipeline, Single steps for different hardware

Kubeflow, MiniKF. Choose Deployment Name in Cloud Console, Kubeflow-kale

Code - Link

Happy Learning!!!