Good Read - Kubernetes - Basics - Connect the Dots

Basic Kubernetes building blocks.

• Containerization (like Docker) - Dockerization, Runtime, Commands, Code - Build Image
• Cloud Basics - Managed solution kubernetes cluster, Identity / Access. Cloud managed solution. Monitor / rebalance pods. Spot instance automation. Optimization features
• YAML - Write config files, Resources in Kubernetes, Desired state of kubernetes deployment object
• Networking Basics - pods communication, clusterip, node ports
• Linux - Command line kubectl tools proficiency, vi

CAST AI is the leading all-in-one platform for Kubernetes automation, optimization, security, and cost management. 

Keep Exploring!!!

Scaling Applications

We have AWS Lambda, GCP Cloud run servless function options. This will help effectively to autoscale.

For custom apps / rest / flask / fast end points how to we auto scale

• Horizontal Pod Autoscaler (HPA):adjusts the number of replicas of an application.
• HPA is a form of autoscaling that increases or decreases the number of pods

Ref - Link

HorizontalPodAutoscaler Walkthrough

Key Notes

kubectl autoscale subcommand, part of kubectl, that helps you do this.

kubectl autoscale deployment php-apache --cpu-percent=50 --min=1 --max=10

# You can use "hpa" or "horizontalpodautoscaler"; either name works OK.

kubectl get hpa

kubernetes-fastapi

How to Test Autoscaling in Kubernetes

Keep Exploring!!!

Kubernetes for 90's Kids

• Essentially it's a way to run containerized workloads
• Production-grade container orchestrator

So What is a Container?

• A container image is a ready-to-run software package, containing everything needed to run an application
• By design, a container is immutable: you cannot change the code of a container that is already running

How do all these things work?

Key Components of Kubernetes cluster

• API Server - interaction point for all Kubernetes components 
• Kubelet - The on-host agent that communicates with the API server 
• Controller Manager - A set of controllers
• Scheduler - Determines where workloads should 
• Kube Proxy - Implements Kubernetes services providing virtual IPs 

So essentially environment to run a packaged application with all binaries and required packages. It handles deployment, monitoring, scaling in the cluster.

Keep Learning!!!

Weekend Learning - Introducing KFServing: Serverless Model Serving on Kubernetes

KF Serving

• Serving story for Kubeflow
• The concepts behind Kubernetes

• Serving Frameworks (Seldon core, ml lambdas, Tensorflow Serving)

• Consistent interfaces for different frameworks

• ServiceAccount for access
• Canary - 2 way split (New / Old)
• Default Standard Deployment
• Canary - Addressable primary / default
• Experimental traffic handling
• Similar to A / B Testing

• KFServing for ML Problems
• Knative - Resource Model
• Production Features of ML KFServing 0.2

• Use cases at Bloomberg

• Serving models in production
• Scaling and handling traffic
• End to End implementation/scalability and load handling

• Model production requirements
• A lot of out of box features for production-grade implementation

• Bloombergs use cases
• All data / NLP

KF Serving Transformer concept

• Implement pre and post processing
• Add transformer to inference service

Model Explanation

• Alibi library

• Accessibility to prediction URL

Kafka Implementation Example
A / B Testing Approach
CI / CD Pipeline

MNIST kfserving

• Preprocess / postprocess in transformer
• Download image
• Run prediction
• Result upload to bucket
• Custom model to process
• Upload to bucket

More Reads
Link1
Sample for KFServing SDK with a custom image
Predict on a InferenceService using Tensorflow

Happy Learning!!!

Day #337 - Lessons Learnt in Playing with Docker, Kubernetes, VOLUME in Docker vs VOLUME Kubernetes

Tip #1 - To make flask API multithreaded, Add the option threaded in app.run()

app.run(host= '0.0.0.0',port=4321,threaded=True)

Tip #2 - To run a flask API in Docker, which involves file operations we can specify VOLUME

VOLUME inside docker file definition - Basically with VOLUME command you get performance without externally mounting any volumes.

VOLUME ['/datadir']

Some good example use cases:

• logs
• temp folders

Ref - Stackoverflow Answer

Tip #3 - Alternate option to #2 - Provide volume while running docker

docker run -p 8080:8080 Demo-App --volume=/Users/tmp:/app Demo-App

Tip #4. Specify Volume - emptyDir inside Kubernetes for temporary data

Use emptyDir Option. emptyDir volume should NOT be used for persisting data. Data erased when pod is removed.

Happy Learning!!!