Forecasting Setup MVP to Large Scale Deployment

Phase I (Initial use cases)

• DVC - Data Versioning and Control
• Databases - Sales / Historical data
• External Marketing Campaign data 
• Feature Engineering - Database / Feature Store
• Model Building - ML / DL Algos
• Model Experiment Tracking - MLflow
• Drift / Monitoring - Evidently AI
• Model Deployment - API / Serverless function
• Actuals Tracking Loading - Database
• Model Scheduling / ETL Scheduling - Jobs based on AWS / GCP / Custom scheduler cron jobs
• I prefer dockerizing it and deploy it in GCP APP engine like a Nocode / Low code approach for the first few models
• Custom Reporting for trends/patterns making it more accessible/relatable to business
• Reporting for Past - Present - Future. Making predictions relatable
• Explainable AI to compare predictions vs actuals to interpret cause - -reason in a more Non-ML Approach

Phase II (Serving large models > 30)

• Kubernetes based Platform
• Once the platform is deployed you can leverage out of box images / notebook / pipeline / monitoring options available
• Kubeflow Pipelines

Kubernetes as a Service: GKE vs. AKS vs. EKS

Build a Pipeline

Keep Exploring!!!

ML Model Deployment Strategies

Real time - API as Service

• Google Assistant speech recognition
• Twitter feed

Streaming Approach

Ref - Link

Batch Prediction

Examples

• TripAdvisor hotel ranking
• Netflix recommendations

Hybrid Approach

Door Dash

• Restaurant recommendations use batch predictions
• Within each restaurant, item recommendations use online predictions

Netflix

• Title recommendations use batch predictions
• Row orders use online predictions

Ref - Link

Model Testing

• Canary testing - New model alongside existing system. Slowly increase the traffic to new model
• A/B testing - New model alongside existing system. A percentage of traffic is routed to new model based on routing rules
• Interleaved experiments - Especially useful for ranking/recsys. Take recommendations from both model A & B. Mix them together and show them to users
• Shadow testing - New model in parallel with existing system. New model’s predictions are logged, but not show to users

Ref - Link

Production ML

Ref - Link

Keep Exploring!!!