Amazon Robotics Routing in FC

• Directions provided by cloud-based algorithms 
• Grid of encoded markers
• Virtual and physical barriers restrict their interactions with people, as well as where they can and cannot go.
• Data collected with the robot’s cameras and LIDAR
• Layered on top of the semantic understanding are predictive models that teach the robot how to treat each object detected

Item fulfillment link 

Keep Exploring!!!

Supply Chain use case - Automating damage detection

• Damage detection is important because while damage is a costly problem in itself, it becomes even more costly the longer the damage goes undetected.
• Identify and fix damaged products before dispatching them to customers. 
• Damage is both heterogenous — any item or product can be damaged — and can take many forms, from rips to holes to a single broken part of a larger set.
• Dataset - 30,000 product images in this way, two-thirds of which were images of damaged items.

Ref from post 

Keep Exploring!!!

Vision for Product Catalog Image Correction

Do you know for each category near / far view has an impact on click-to-view conversion

Challenge - How do we know similar products?

Solution - Build vectors of text for each category/features/images. measuring distances between vector representations of products in an embedding space. Features for image vector - product image and the product title

Challenge - How do you change/customize catalog?

Solution - The title provides context that helps the model focus on the relevant regions of the image. Based on it remove the background / zoom  it

Challenge - How does the solution look like? How many models?

Solution -  Global network takes the whole image as input and based on the product title, it determines which portion of the image to focus on. That information is used to crop the input image, and the cropped image passes to the local branch.

From Ref - Using computer vision to weed out product catalogue errors

Keep Exploring!!!

Solving #Vision Problem - #Amazon Way - Object Detection

To detect objects in Conveyer, How does Amazon Solve it?

Challenge - Field Of View, Dataset, and Occlusion is always a challenge to detecting objects

Solution - The first step was simply to take pictures of products as they moved along conveyor belts in fulfillment centers, building up a library of images. This gives a consistent environment

Challenge - Different speeds of products moved in the Conveyer belt?

Solution - On a conveyor belt, the lighting and the speed of the item are relatively controlled and constant.

Challenge - What features we can leverage to match?

Solution - Product Dimension, Visual features, text

Summary from post

#Datascience #production #computervision 

Sometimes a combination of techniques, simplifying/restricting the target environment to a certain position/view/conditions is good enough to get a good first-cut working solution.

Keep Exploring!!!

Generative AI Startups

 Ref - From Link

Keep Exploring!!!

Learning Process

Building Solutions #Datascience #perspectives

As we near challenging moments, a lot of papers/research / State of Art #SOTA Discussions. Creating Solutions from it requires.

• Try new #tools often, especially ones that use a different approach
• Need to blend concept + value proposition. #Conceptual clarity - What are the basics, and How it works. #Logical thinking - What are solutions/options, beyond the paper to real-world applications, and how it applies to implementation.
• Copy-paste code knowledge is not true knowledge, #Production grade Solutions vs Demo both are different levels, even if people pitch both at similar levels as projects.
• Learning is a repetitive process, Learning is connected to aspects of related topics. Learning needs frequent experiments / corrections and updates
• Along the way, ML is joined by MLops, Cloud Stack, and Deployment, Serving a few more siblings to make the journey more research focused
• Ship early, Ship often, and Refine it more often. 

Keep Exploring!!!

Five things About GPT

Five Things about GPT

#1. GPT-3, or the third generation Generative Pre-trained Transformer, is a neural network machine learning model trained using internet data to generate any type of text

ChatGPT is a chatbot technology developed by 𝐎𝐩𝐞𝐧𝐀𝐈. It is designed to assist with a variety of tasks and functions, including answering questions, providing information, and completing tasks. 

#2. Number of layers and parameters

#3. Parameters

• GPT-2 was released in February 2019 with 1.5 billion parameters 
• GPT-3 was released in June 2020 with 175 billion parameters (~120x improvement)
• GPT-4 will be released soon and is expected to have 100 trillion parameters (~500x improvement)

#4. How intelligent can it be?

#5. GPT-4 is built on the Transformer architecture, which has been effective for a variety of machine-learning tasks, including computer vision. This means that GPT-4 might be used for tasks such as image and video generation

#6. Training Approach

Keep Exploring!!!

ML Model Deployment Strategies

Real time - API as Service

• Google Assistant speech recognition
• Twitter feed

Streaming Approach

Ref - Link

Batch Prediction

Examples

• TripAdvisor hotel ranking
• Netflix recommendations

Hybrid Approach

Door Dash

• Restaurant recommendations use batch predictions
• Within each restaurant, item recommendations use online predictions

Netflix

• Title recommendations use batch predictions
• Row orders use online predictions

Ref - Link

Model Testing

• Canary testing - New model alongside existing system. Slowly increase the traffic to new model
• A/B testing - New model alongside existing system. A percentage of traffic is routed to new model based on routing rules
• Interleaved experiments - Especially useful for ranking/recsys. Take recommendations from both model A & B. Mix them together and show them to users
• Shadow testing - New model in parallel with existing system. New model’s predictions are logged, but not show to users

Ref - Link

Production ML

Ref - Link

Keep Exploring!!!

Model Optimization / Performance Key Notes

• Knowledge distillation is a method in which a small model (student) is trained to mimic a larger model or ensemble of models (teacher). 
• DistilBERT, reduces the size of a BERT model by 40%, while retaining 97% of its language understanding capabilities and being 60% faster.
• Pruning was a method originally used for decision trees where you remove sections of a tree that are uncritical and redundant for classification
• The more common meaning is to find parameters least useful to predictions and set them to 0
• Quantization reduces a model’s size by using fewer bits to represent its parameters. 
• By default, most software packages use 32 bits to represent a float number (single precision floating point). If a model has 100M parameters, each requires 32 bits to store, it’ll take up 400MB. If we use 16 bits to represent a number, we’ll reduce the memory footprint by half. Using 16 bits to represent a float is called half precision.

Knowledge Distillation

Pruning in Keras example

Ref - Link 

Ref2 - Link 

Large Transformer Model Inference Optimization

• Reduce the memory footprint of the model by using fewer GPU devices and less GPU memory;
• Reduce the desired computation complexity by lowering the number of FLOPs needed;
• Reduce the inference latency and make things run faster.
• Post-Training Quantization (PTQ): A model is first trained to convergence and then we convert its weights to lower precision without more training
• Unstructured pruning is allowed to drop any weight or connection, so it does not retain the original network architecture.
• Structured pruning aims to maintain the dense matrix multiplication form where some elements are zeros

Keep Exploring!!!

Model Training Checklist

Often I end up running a working code but poor segmentation results. Need to save cost / try low res to slowly make changes.

Making a working model on a limited set is key.

• Run for limited set 1000 images
• Run for smaller resolution 128x128
• Run for smaller batches
• Model Compilation
• Batch Compilation
• Adjust learning rates

Keep Exploring!!!