"No one is harder on a talented person than the person themselves" - Linda Wilkinson ; "Trust your guts and don't follow the herd" ; "Validate direction not destination" ;

February 27, 2020

Evolution of Data Storage, Analysis, Analytics - Database - DW - DataLake

2000 - 2010
  • Stage 1 - Papers, Ledgers
  • Stage 2 - Excel, Access
  • Stage 3 - Databases for OLTP / DW for OLAP (ACID Properties)
Knowledge -  I know how my business is performing as of today. I know the past 6 months of historical data and performance

2010 - 2015
  • Stage 4 - Hadoop for large scale DW (Velocity, Volume, Veracity)
  • Stage 5 - NoSQL (CAP Theorem)
Knowledge -  I store all possible data without strict data type validations, can query in large scale data for Adhoc queries. Schema on Read, Finding insights from unstructured data (logs, text, records, events)

2015 - 2020
  • Stage 6 - AI to extract  interpretable data from Image, Video, Sound (Numbers, Object classification, Count, etc)
  • Stage 7 - lakehouse = (Hadoop + RDBMS + NoSQL + AI for data extraction from unstructured sources)
Knowledge - I have insights from structured, unstructured, vision, audio and every type of data. Handling all types of data and finding meaningful insights.

CAP Properties and Databases


More Reads
Spanner Reads

Happy Learning!!!

No comments: