GCP Experiments

• Retail interactive tutorials
• Vertex Pipeline
• MinkKF
• KF Pipeline
• MiniKF Kale

Keep Exploring!!!

Python + Data Pipelines

Data fetched from multiple sources

Data Integrity purity test

Lessons

ETL vs ELT

DAG / Pre-requisites, Remember order of sequence

• Many ordering would be there
• CDC will have issues

Lesson

Longer bad data more cost to pay

Backward compatible support schema

Keep Exploring!!!