Day #219 - Person Re-Identification

All the research papers with code available in site paperswithcode 
Reid Resources 

Overall Lessons

• CNN based auto encoders to encode an input image, and then using K-nearest neighbor algo, find the closest match to the encoded images in a database
• Query2Gallery Similarity using Euclidean distance
• Foreground, Head, Upper Body, Lower Body used for Cues
• Detection + Classification
• Local Maximal Occurrence (LOMO) analyzes the horizontal occurrence of local features, and maximizes the occurrence to make a stable representation against viewpoint changes
• Video tracklets in person re-identification

Key Lessons
Talk #1 - Human Semantic Parsing for Person Re-identification

• Query Image
• Retrieve all images of the same identity
• Query, Top 10 Retrieved Matches

Challenges

• Illumination Condition
• Background Clutter
• Occlusion
• Observable Body parts not visible
• Hard to obtain posture
• Extracting Robot visual representation
• Low-Resolution Images

Questions

• Develop complex models?
• Extract Local Visual Cues?
• Human Pose Estimation used to estimate 
• Unable to identify arbitrary contours of body parts
• Methods of Horizontal stripes

Contributions

• Human Semantic Parsing (SPReid)
• Simple holistic models work

SPReid

• Inception-V3 architecture
• Modified Inception-V3 architecture
• Dilated Convolution

Architecture

• Image - Inception V3
• Avg Pooling get final representation
• Foreground, Head, Upper Body, Lower Body used for Cues
• One Global
• One Foreground

Training and Evaluation

• Softmax cross entropy loss
• Train on low resolution, fine tune on high resolution
• Look into person (Dataset)
• Query2Gallery Similarity using Euclidean distance

Talk #2 - Joint Detection and Identification Feature Learning for Person Search | Spotlight 2-2B

Key Lessons

• Match Photo with Manually Crafted
• Find from the whole image, Detect People and Extract People and Features
• Softmax classifier
• Detection + Classification
• Online instance Matching
• Labeled One's Lookup Table
• Minimize the distance between sathe me person

Talk #3 - Unsupervised Person Re-identification by Deep Learning Tracklet Association

Key Lessons

• Supervised (Pairwise Neighboring)
• Triplet Loss
• Manually Labelled, Impose huge constraint
• Completely Unsupervised using tracket associations
• Collect Tracklet Data
• Tracklet Sampling
• Tracklet Association
• Histogram Loss, Surrogate Loss

Siamese Network
Key Lessons

• Find similar faces
• Sequence of CNN, Pooling and Feature vector
• Fed to make classification
• Number computed vector F(x1) - Encoding of input Image
• Feed second pic and get another F(x2)
• Encoding is good representation, Find distance between x1 and x2
• Two CNN and comparing them is Siamese Network Architecture
• Train NN that generates encoding

More Reads
One Shot Learning with Siamese Networks using Keras
Image Similarity with Siamese Networks
Keras Example1
Siamese Network
Survey on Deep Learning Techniques for Person Re-Identification Task
Unsupervised Person Re-identification by Deep Learning Tracklet Association
Enhanced Deep Feature Representation for Person Re-identification
WACV18: Vehicle Re-identification by Adversarial Bi-directional LSTM Network

Survey on Deep Learning Techniques for Person Re-Identification Task
Key Notes

• On-line applications for people/object detection and tracking
• Recognizing a suspicious action/behavior from the camera network
• Off-line applications to support operators and forensic investigators 

Image Challenges

• Low image resolution
• Unconstrained pose
• Illumination changes
• Occlusions 

Features to Exploit

• Face
• Clothing appearance
• Gait
• CNN generates a set of feature maps in which each pixel of given image corresponds to a specific feature representation
• Image Size - 128 × 64

DNN Key Considerations

• Objective function
• Loss functions
• Data augmentation

Feature fusion deep neural network

• Network takes a single image size of 224 × 224 × 3 as the input of the network
• Hand-crafted features are extracted by one of the standard person re-identification descriptor
• Both extracted features are followed by a buffer layer and a fully connected layer which are acting as the fusion layer
• A softmax loss layer then takes the output vector of fully connected layer in order to minimizing the cross-entropy loss

Siamese network

• Siamese network models have been widely employed in person re-identification task
• Employed as pairwise
• Two subnetworks included
• Output is similarity score

Tripletmodels
Training sample separately fed into three identical networks with shared parameter set between them
For each triplet unit they organized to maximize the margin between the matched pairs and the mismatched pairs. Hinge loss, Cosine similarity loss, Contrastive loss

Happy Mastering DL!!!!

My Career Journey

2003 - Testing windows OS is great
2004 - Coding C++ MQ adapter is interesting
2005 - Setting up an Application support team and Swiss onsite
2006 - Finding my place in Microsoft, Learning the Domain, Supply Chain
2007 - Performance, SQL Migration, Biztalk and Automation
2008 - DB Developer
2009  - BI / OLTP performance tuning / TSQL developer
2010 -  BI / OLTP performance tuning / TSQL developer
2010 - Need more $$ and challenges
2011 - Setting up Team in Amazon
2012 - Better become Individual Contributor, Again Database and QA, Setting up Team
2013 - Performance, Automation, Database development
2014 - Big data jump and getting into it
2015 - Start from scratch Data science
2016 - Data Science year 2 All 2 year Masters deep dive
2017 - Data Science year 3 Image and Data Analytics projects
2018 - Data Science year 4 Machine Learning projects
2019 - Data Science year 5 Deep learning Projects
2020 - Vision Expert and Deploying Solutions in Scale

To sum up - "Data Guy, Empirical Learner"

The days I spent weekends learning is more than my weekday efforts. I have witnessed the transformation of technology and the tools that evolved all these years. Experimenting and working on different roles, starting things from zero and re-learning has been a rewarding experience. All these years I have learned, relearnt, and still learning. Outside this life also gave me lessons and blessings.

Keep learning, Keep growing!!!

Big Data and SQL Journey

I have been a silent observer of Big Data Journey.

• In 2010 - Ran SQL Queries on RDBMS for Committed Data
• Entity Framework came into the picture, heated debate Stored procedures are no longer needed
• Hadoop came, I queried on HBase, Hive for the Real-time / Analytics queries for Current Data and Historical data
• After a few years now I see the queries in Spark SQL and KSQL for Streaming Passing Data
• SQL queries are shifting all the way from Database - Real-time - Real-time Stream Querying
• SQL Skills are coming to every layer of the data processing stack

Querying now goes for - incoming data, passing data, current data, historical data

• Kafka SQL - SQL for Incoming data
• Spark SQL - SQL for Passing data
• HBASE - SQL for Real-time data
• HIVE - SQL for historical data

All Hail SQL!!!

Happy Learning!!!

Day #218 - Working on gluoncv, anaconda and Windows 10

In my base Anaconda 3+ Environment,

pip install gluoncv
pip install mxnet

Download model from https://modelzoo.co/model/ssd-mxnet, This step was not required, Figured out later this was due to below package issues scipy and matplotlib.

Error in Call back
pip uninstall scipy
pip install scipy
pip uninstall matplotlib
pip install matplotlib

Below is example code

Data gets downloaded to location
C:\Users\#user#\.mxnet\models

The Output example is

Happy Mastering DL!!!

Day #217 - OpenVino Session

Key Lessons

• Open Visual Inference Image and Neural Network Optimization Tool Kit
• Tools & Capabilities for Developers across domains
• OpenVino is supported only for Intel Devices
• Models for purpose
• Models to detect across the frame
• Hardware for Performance
• Compute Efficiency / Memory Hierarchy / APIs
• CPU and Integrated FPGA, GPU solutions

Pipeline flow

• Decode compressed Image
• Preprocessing - Scale down for DL model input, ROI computation, Frame Re-ordering
• Post-Processing - Write bounding boxes on top of it

Inference

• Training offline activity
• Model Optimizer would do conversion for CPU, GPU, FPGA
• Intel Library also added on it
• Out of Box Models in OpenVino
• Compile for Target
• mo.py model optimizer FP16. xml and bin file generated
• Movidus neural compute stick
• -d cpu, -d gpu, -d myriad
• Use of Hetro Plugin - GPU and CPU
• -d HETRO:GPU,CPU

More Reads - 
https://github.com/intel-iot-devkit/store-traffic-monitor
https://github.com/intel-iot-devkit/smart-video-workshop



Happy Mastering DL!!!

Day #216 - My Date with OpenVino

The Steps are detailed in link

On Running the following Pre-requisites were mentioned. 
1. OpenCL Driver - Intel Driver Update Utility.
2. Visual studio 2017
3. Python 3.6 64 bit

Followed Steps provided on VS2017 Package and Details
conda install python=3.6.5

After installation goto the environment and run
1. Goto Folder and Run C:\Intel\computer_vision_sdk\bin\setupvars.bat

2. Run C:\Intel\computer_vision_sdk_2018.5.456\deployment_tools\model_optimizer\install_prerequisites

3. protobuf not installed error
4. pip install protobug

5. Re-Again - demo_squeezenet_download_convert_run.bat

6. Run Next Demo demo_security_barrier_camera.bat
7. Output is the image

After the Output seems my system and myself are on the same page. The date is Over!!!

Available Models - C:\Intel\computer_vision_sdk_2018.5.456\deployment_tools\computer_vision_algorithms\share\cva

Models located in C:\Intel\computer_vision_sdk_2018.5.456\deployment_tools\computer_vision_algorithms\share\cva\PersonReidentification\doc\examples

Outside the build file it builds executable and inference engine
C:\Users\#user#\Documents\Intel\OpenVINO\inference_engine_samples_2017\intel64\Release\classification_sample.exe -i C:\Intel\computer_vision_sdk_2018.5.456\deployment_tools\demo\\car.png -m "C:\Users\#user#\Documents\Intel\OpenVINO\openvino_models\ir\FP32\classification\squeezenet\1.1\caffe\squeezenet1.1.xml" -d CPU

There is a build file which needs to be understood and built to fix it. I was quickly trying to leverage the exe.

More Examples - https://software.intel.com/en-us/articles/OpenVINO-IE-Samples#multi-channel-face-detection-sample

Happy Mastering DL!!!

Spark Lessons #Applying Best Practices to Your Apache Spark Applications - Silvio Fiorito

Key Lessons

• Spark is lazily executed
• Apply transformations to query
• Count, Write, For Each
• Reader API - Spark.read.load

• Class - InmemoryFileIndex (Responsible for partition discovery) - S3 / HDFS

• Anything over 32 folders will kick off job
• Dealing with Many partitions
• InMemoryFileIndex to index paths you are interested in

Datasource Tables

• Managed in Hive Metastore

• External or unmanaged tables (Hive Schema over existing Dataset)

• Managed Table (SparkSQL Manages)
• Hive also keeps track of schema
• Files / Tables diff

• Tables - Schema in Metastore
• For BI users you can use Tables
• Dealing with CSV / JSON files
• Scan dataset and creates schema - convenient for slow dataset (schema inference)

Compression and Partition Scheme

• Depends on Adhoc / Batch
• Splittable compression schemes
• Avoid Large Gzip text files

Optimization

• Partitioning / Bucketing (persist hash partitioned data, good for joins and keys)
• Each task will write a file in the bucket

• Repartition by partition by value / column
• One File per partition

Query Optimization

• SQL shuffle partition
• Default value override based on data volume
• Self-Union Reading dataset twice
• Cost based optimizer

Happy Learning!!!

Big Data Lessons #Lessons learned form Kafka in production (Tim Berglund, Confluent)

Key Lessons
Events

• Data has to go into the database
• Kafka - All of your data is events stream
• Kafka is having an opinion of the world

Sensors

• Sensor data are events
• Car companies with Internet Connected Devices
• Log Entries are an operational thing
• Logs are events

Databases

• Databases can also be events
• Table - Collection of Key-value pairs
• Modifications as messages
• Updates can be stream of messages

Uses of Stream

• Data Pipeline
• React / Process / Transform

Event Centric Thinking

• Web App -> Streaming Platform -> Hadoop
• "Product View Request"
• Forward Compatible (Receive Requests from Multiple Interfaces
• New services can listen and easy to extend the system to evolve the system going forward

Kafka Overview

• Producers - Kafla CLusters - Consumers
• Data Model - Log
• Write comes at the end
• Log file
• Multiple consumers can read from the log
• Reader - Consumer
• Writer - Producer
• Kafka Topic = Partitioned Log
• Kafka is Distributed Message Queue
• Each Topic is a partitioned Log
• Partitioned among multiple computers (Brokers)
• The producer decides partition to write to
• Kafka, we have ordering within the partition
• Ordering within the partition but not available globally
• There is no global ordering
• Table and stream are isomorphic
• Group of Consumers
• Consumer groups handy way to divide among multiple consumers

Scalability of File System

• Write Part
• Read Part
• Indexes, Merge, Log Tree

Kafka

• Hundreds of MB / Sec throughput
• Commodity hardware
• O(1) writes

Distributed by Design

• Replication
• Fault Tolerance
• Partitioning
• Elastic Scaling

Issue #1 - Strange Happenings with Partitioning

• Partitioning
• One Lead Partition
• Multiple Followers
• One broker acts as a controller
• Partitioning is to scale a Topic
• Leader and Follower Partitions
• Four Partitions and Three Replicas
• ISR - In Sync Replica - Caught up with Leader
• For a write to be committed it has to be commited by the leader and all other ISR
• Watch your ISR List
• Upgrade all the brokers in Rollout Fashion (Keep them in the same version)

Issue #2 - Automated Liveliness check

• Broker Kept Failing
• Leader Failure
• More Partitions more throughput
• More partition longer to balance cluster
• Router Configuration problem

Issues #3 - Adding a Broker Hurts

• Custom Environment
• Kafka Reassignment Partition Tool
• Generate Migration Instructions

Happy Learning Best Practices!!!

NLP One Day Training Map

Happy Mastering DL!!!

Personalized Travel Plans

I found this company Pickyourtrail interesting. I wanted to analyze more on their personalization algorithm. Some thoughts on the same.
Product Concept - Personalized End to End Travel Plans
Selling Point - Personalized recommendations
Target Audience - Well to do earning professionals

The initial dataset they collected on Travel Destinations, Trending destinations, Historical data is key

If we have to take an Automated Recommendation using ML Algorithm. I would see it this way

• Collect Social Media Data
• Analyze Income Range 
• Historical Data from previous travel collected from Social Network
• Interests collected from Likes, Comments from Social media
• Segment the customers into categories - Wildlife Travel, Spiritual, Normad Trips, Historical Interests
• Match their months of previous travel
• Match based on the data collected on future interests

Possible Options - Banks can tie-up and provide such offers. Instead of credit points personalized travel plans with tie-up from these vendors
Futuristic Way - Everything tied Banks - Flight Plans - Uber - Airbnd, Everything connected end to end and adjusted based on time changes and variations

Happy Learning!!!