October 17, 2023

Python + Data Pipelines

Data fetched from multiple sources

Data Integrity purity test

Lessons

ETL vs ELT

DAG / Pre-requisites, Remember order of sequence

  • Many ordering would be there
  • CDC will have issues



Lesson

Longer bad data more cost to pay


Backward compatible support schema


Keep Exploring!!!

No comments:

Post a Comment