Streamlit Deployment Notes examples

•  30 Days Streamlit learning
• Deploy streamlit to Google App Engine in the flexible environment without any docker/container.

runtime: python
runtime_config:
operating_system: “ubuntu18”
runtime_version: “3.8”
entrypoint: streamlit run --server.port=8080 --server.address=0.0.0.0 --server.enableCORS=false --server.enableWebsocketCompression=false --server.enableXsrfProtection=false --server.headless=true app.py
env: flex
network:
session_affinity: true

• ViLT-GPT
• face_detection_app
• NLP Based App with Streamlit
• Deploy Streamlit using Docker
• Deploy Streamlit using Kubernetes
• Ask my PDF
• Langchain - Text Summarization App
• Unlock your creativity with LLMs

Keep Exploring!!!

Model Deployment Architecture

 My implementation experience and lessons :)

Product Implementation (2012-2014)

• Integrated in product
• Jobs scheduled for midnight
• Workflow to monitor variations
• Forecast updated every day for store
• Everything custom-coded formula embedded
• Weighted moving average 
• Step up / Step down moving average approach

Batched State of Art (2021)

Recommendations AWS

• ETL / Glue jobs to get featured
• Full pull/delta pull scripts
• Feature engineering scripts
• Custom segmentation scripts
• Batch jobs to run models
• Large-scale recommendations generation
• Infra kubeflow setup 
• Leverage existing Kubeflow monitoring setup

Forecasting State of Art (2021)

Kubeflow + AWS

• ETL / Glue jobs to get features
• Full pull/delta pull scripts
• Feature engineering scripts
• Custom segmentation scripts
• Batch jobs to run models
• Kubeflow pipelines for the forecast
• Results persist in Redshift DB
• Infra kubeflow setup 
• Leverage existing kubeflow monitoring setup

Realtime State of Art (2022)

Real-time streaming / Vision Solution

• AWS Lamdbda-based approach
• Vision + Docker + AWS Lambda
• Request monitoring / logging

Keep Exploring!!!

Model Deployment Architecture - The two sides of coin

 My implementation experience and lessons :)

Product Implementation (2012-2014)

• Integrated in product
• Jobs scheduled for midnight
• Workflow to monitor variations
• Forecast updated every day for store
• Everything custom-coded formula embedded
• Weighted moving average 
• Step up / Step down moving average approach

Batched State of Art (2021)

Recommendations AWS

• ETL / Glue jobs to get featured
• Full pull/delta pull scripts
• Feature engineering scripts
• Custom segmentation scripts
• Batch jobs to run models
• Large-scale recommendations generation
• Infra kubeflow setup 
• Leverage existing Kubeflow monitoring setup

Forecasting State of Art (2021)

Kubeflow + AWS

• ETL / Glue jobs to get features
• Full pull/delta pull scripts
• Feature engineering scripts
• Custom segmentation scripts
• Batch jobs to run models
• Kubeflow pipelines for the forecast
• Results persist in Redshift DB
• Infra kubeflow setup 
• Leverage existing kubeflow monitoring setup

Realtime State of Art (2022)

Real-time streaming / Vision Solution

• AWS Lamdbda-based approach
• Vision + Docker + AWS Lambda
• Request monitoring / logging

Keep Exploring!!!

EC2 - AWS Flask Deployment Example

Deploying Sample API code 

Creating an EC2 Free t2 instance

Basic Ubuntu

Allow Access

Save Public IP

Launch Instance

Run Updated Packages and the below commands

Ref - Simplified code example Link

Allow Access to Port 80. HTTP

Run from browser and access APIs

Keep Exploring!!!

ML Model Hosting, Deployment

ML Model Hosting
This post is to discuss different techniques for Model hosting. The real-world scenario will involve

• Data Pipeline
• Model Training
• Model Deployment (Saved File / Saved Coefficients and compute at runtime)
• Model Rendering
• Framework for monitoring model performance
• Model Retraining

Data pipeline would be the data obtained from transactional / BI systems
Model Training would involve pre-processing, normalization, bucketing and multiple features. This is like metadata. This needs to be stored in a metadata table or a separate standalone script to apply for the test data
Model deployment - Save the model as pickle file / Save the coefficient values in DB with metadata value to compute
Model Rendering - Exposure as a Service, Invoke this after a service call
Monitoring model performance - Expected / Actual values need to be monitored, deviation beyond a certain threshold would need human in loop to update / re-train / fix the model
Model Retraining - Model has to be updated on a regular basis or deviation from certain threshold accuracy

Tools -
Flask API, Web Interface, Visualization of Clusters, Recommendations, SQL Code/ Python code for preparing feature variables, Handling Missing data, normalization (standard code), Logging, Model monitoring schema

Successfully augmenting human expertise with Artificial Intelligence and other smart technologies starts with the person at the top - How CEOs can tap AI’s full potential. Link >> https://t.co/1dy8VdrmMv @PwC via @antgrasso @MikeQuindazzi #AI #Automation #DigitalStrategy pic.twitter.com/Js8Q0Bc1mj

— Antonio Grasso (@antgrasso) May 27, 2020

References
Click Prediction ML Software Pipelines from the Trenches
Architecting a Machine Learning Pipeline
Machine Learning Pipeline from scratch
Models to Production
How to Code Neat Machine Learning Pipelines
Lessons learnt while building a Machine Learning Pipeline
Cookiecutter Data Science
production-data-science

Happy Learning!!!