"No one is harder on a talented person than the person themselves" - Linda Wilkinson ; "Trust your guts and don't follow the herd" ; "Validate direction not destination" ;

November 24, 2017

Database Sharding and Scalability Basics

Some Key considerations for NOSQL Vs RDBMS
  • Performance - Latency tolerance, How slow my queries can run for huge data sets
  • Durability - Data loss tolerance when database crashes losing in-memory or Lost transactions tolerance
  • Consistency - Weird results tolerance (Dirty data tolerance)
  • Availability - Downtime tolerance
Options for Scalability
  • Replication - Create copies of database, Application can talk to either database
  • Sharding - Sharding choosing a partition key, Key-value store partition based on key
  • Caching - Precomputed and stored, Manage cache expiration time and refresh logic
For streaming data we had already discussed Events Hub, Apache kafka. Now we have something called KSQL (Kafka streaming SQL to run on continuous data)

Great Session Talk

 

RDBMS VS NOSQL Considerations, Quick Summary
  • Performance - Latency tolerance
  • Durability - Data loss tolerance
  • Consistency - Weird results tolerance (Dirty data tolerance)
  • Availability - Downtime tolerance
Happy Learning!!!

No comments: