Kubeflow Pipelines - Learning Notes #1

To appreciate something we need to why, how, what about the Tool.

Key Notes

Why / Necessity

1. Monitoring of Model
2. Training /Serving - Differences in transformation, handling missing data
3. Frequency to refresh the model

Production System Components

Kubeflow Platform

Develop, Deploy, Manage

Pipelines, Data Management, Serving (Rest End Point) 

Pipeline Component

Commands

Setup cluster, permissions in yaml file

Demo with screenshots

Pipelines

• Domain-Specific Language
• Instantiate Components
• Define Dependency between components
• Compile and Deploy Pipeline

Custom Components

Somehow the gap between ML code vs kubeflow code there is a lot of learning. How much time it takes to port to this infra? I need to experiment to comment. A lot of features are there but we shouldn't end up rewriting ML code to pipeline code. 

Notes #2

Codify ML Workflows

Adopt pipeline mindset

Experiment, Reproduce, Share pipeline

Define Pipeline

• The description on ML Workflow
• Runs on Container
• Execution vs Runtime decoupled
• Components - one step of workflow
• Component - Packaged as Docker image
• Pod for Each Step
• Pipeline SDK

More Reads

Example

pipeline sdk key notes - Link1, Link2, Link3

SDK Summary pointers

KALE (Kubeflow Automated pipeLines Engine) is a project that aims at simplifying the Data Science experience of deploying Kubeflow Pipelines workflows.

An Argo workflow executor is a process that conforms to a specific interface that allows Argo to perform certain actions like monitoring pod logs, collecting artifacts, managing container lifecycles, etc.

Katib is a Kubernetes-native project for automated machine learning (AutoML). Katib supports hyperparameter tuning, early stopping and neural architecture search (NAS). Learn more about AutoML at fast.ai, Google Cloud, Microsoft Azure or Amazon SageMaker

Katib is the project which is agnostic to machine learning (ML) frameworks.

Ray Train, an easy-to-use library for distributed deep learning.

Dask is a flexible library for parallel computing in Python.

ML metadata (MLMD) library by Google. MLMD is an integral part of TensorFlow Extended (TFX) and a stand-alone application

The most important entities created and stored by MLMD are:

• Artifacts that are generated by the pipeline steps (e.g., the trained model).
• Metadata about the executions (e.g., the step itself).
• Metadata about the context (e.g., the whole pipeline).

Keep Thinking!!!

Smartphone / Social Media Issues

There are a lot of issues due to cheap internet smartphone / social media, The primary motive seems to have failed

• Targeted marketing to influence people
• Change from Morality to Majority Opinion is right
• Crowd following mentality
• Drive more consumeristic / purchase behavior
• Focus more on lifestyle/luxury than principles/priorities
• Low or no importance to education
• Setting up wrong examples
• Addiction to games/media / fake gurus
• Commercialize everything, Take away all the time
• In a way they have created isolated hotspots, depressed, addicted to smartphones. With cheap internet, they have only screwed their lives further. Targeted apps for each age group. Drive based on their emotional, behavioral needs. 

Keep Questioning!!!

Interesting Research paper Read - Gender and Race Preferences in Hiring in the Age of Diversity Goals: Evidence from Silicon Valley Tech Firms

Paper - Link

Key Insights

• Women are 9-10% more likely to receive a callback compared to men, 
• Whereas Black Hispanic and Asian applicants are 8-13% less likely to receive a callback compared to White applicants

Key Notes

Studying hiring discrimination at the intersection of race and gender, giving primacy to both

How hiring discrimination, in particular, leads to occupational segregation.

Experiment #1 - Send Fictitious resumes with randomized white-sounding and black-sounding names to potential employers for different types of occupations and consistent discrimination against African Americans across occupations (Bertrand and Mullainathan 2004).

Insights

Statistical discrimination - Employer who imperfectly observes an applicant's quality and productivity resorts to group-level averages to make inferences about the individual, which may lead to discrimination

Taste-based - Employers may have a prejudiced taste and animus towards a particular group, leading to discrimination (Becker 1971).

Discriminatory phenomenon -  female discrimination in male-dominated occupations and male discrimination in female-dominated occupations

ML Approach

Keep Thinking!!!

Personal Datawarehouses

We need to have the ability to claim our personal data / use it to trade without PII. Everything now is paid Gmail / google photos/youtube. The end-user data is used without any benefits for End-user.

• Reclaim your Google Data
• Reclaim your Social Media Data
• Reclaim your Amazon Data
• Reclaim your Location Data

Hope there is some value for user data.

Keep Thinking!!!

Session - ODSC East Talk - Challenges of machine learning development

Code - Link

Key Notes

• Automated and Scalable Infra for ML 

ML Automation Steps

• Data Pipeline
• ML Team
• Production / Deploy / Feedback Mechanism

Key Challenge is integration of all stages of development

• Reproducibility via docker
• Scaling via Kubernetes

Reproducibility

• Share insights
• Deploy Code

Automation process

• Data pipeline
• Feature generation

Production

• Monitoring
• Logging
• Packaging

End to End workflow of Development to Production

Infra 

• Deploy
• Monitor
• Train
• Scale it on cloud

Data Pipeline process

Keep Thinking!!!

Weekend Reads - Advanced Models for Computer Vision

Key Notes

What Classifier will Miss - Human-level scene understanding

• Parsing the scene
• The angle of Bicycle (Pose, Relative pose)
• Person on Bicycle
• Closer Inspection

Tasks

• Object Detection
• Pose Estimation
• Accuracy vs Efficiency of Models

CNN as Deep Learning Puzzle

Input-Output Node, Loss Computation and Backprop

Classification - sparse description of the image

Object Detection

• Multi-task problem
• Classification & Localisation
• Object, Location, Bounding box
• Dataset, Samples, List of Objects, Labels, Bbox for each object

Predict BBOX Coordinates

• Continuous Output
• Minimize mse of samples
• Regression for bbox prediction
• The first part is the classification
• The Second Step is regression

Faster RCNN

• Two-Stage Detector
• Good Candidate BBOX
• Refine through Regression
• Discretize bbox space
• Anchor points distributed
• Candidate boxes of different scale and ratio
• n candidates per anchor
• Is there an object or not in the box
• Refine through regression
• We cannot backdrop on parameters of bbox (Spatial Transformer Networks)

One Stage Detector - Train end to end

• Employ Hard negative mining

Retinanet uses Focal Loss (The loss function is just a mathematical way of saying how far off a guess is from the real value of a data point.). It puts more weight on the objects that were hard to classify and decreases the impact on easy correct predictions

Semantic Segmentation

• Pooling - reduce the resolution of feature maps
• Upsample based on the nearest neighbor approach

U-Net

• Segmenting medical images
• Input Image -> Convolution -> RELU pooling
• Encoder - Similar to Image Classifier
• Upsampling through Decoder for same resolution output
• Upsampling - blobby feature map
• For every location distribution over classes
• Cross Entropy (Avg Over all Locations)

Keep Thinking!!!

Next Paper Read - Docker, RDBMS to ML

Paper #1 - An introduction to Docker for reproducible research

Key Notes

Docker provides a binary image in which all the software has already been installed, configured and tested

Technical Issues in Software Deployment

• Software Dependency Hell
• Imprecise documentation

Docker Features

• Performing Linux container (LXC) based operating system (OS) level virtualization
• Portable deployment of containers across platforms component reuse
• Versioning of container images
• Docker images share the Linux kernel with the host machine
• Sharing the Linux kernel makes Docker much more lightweight and higher performing than complete virtual machines

Components

• Dockerfiles provide a simple script (similar to a Makefile) that defines exactly how to build up the image
• Docker also supports Automated Builds through the Docker Hub (hub.docker.com).

Paper #2 - The Relational Data Borg is Learning

Key Notes

• RDBMS in Data Science
• Widespread need for efficient data processing
• Process beyond classical database workloads
• From the Survey 65% data is Relational. Retail has maximum structured data :)

Automated Feature Learning Approach

Key Features for Retail Stores

• Items in stores
• Store information
• Demographics for areas around the stores
• Inventory units for items in stores on particular dates
• Weather Information

Queries based on Filters

• Feature extraction query that joins these relations on keys for dates, locations, zipcode, and items
• LMFAO (Layered Multiple Functional Aggregates Optimisation) 
• PCA over relational data

Insights

• Running aggregates over days, weeks, months; min, max, average, median aggregates, or aggregates over many-to-many relationships and categorical attributes

ML Tasks

• One-hot encoded
• Categorical attributes
• New database workload motivated by a machine learning application
• Similar aggregates are derived for k-means clustering

(Iterative Functional Aggregate Queries) Framework

• IFAQ can automatically synthesise and optimise aggregates from ML+DB workloads

Key Insights / Lessons

• Turn the learning problem into a database problem.
• Exploit the problem structure to lower the complexity.
• Generate optimised code to lower the constant factors

There is no Data Science without Database - RDBMS :) :)

Happy Learning!!!