Python + Data Pipelines

Data fetched from multiple sources

Data Integrity purity test

Lessons

ETL vs ELT

DAG / Pre-requisites, Remember order of sequence

• Many ordering would be there
• CDC will have issues

Lesson

Longer bad data more cost to pay

Backward compatible support schema

Keep Exploring!!!