Database Sharding and Scalability Basics

Some Key considerations for NOSQL Vs RDBMS

• Performance - Latency tolerance, How slow my queries can run for huge data sets
• Durability - Data loss tolerance when database crashes losing in-memory or Lost transactions tolerance
• Consistency - Weird results tolerance (Dirty data tolerance)
• Availability - Downtime tolerance

Options for Scalability

• Replication - Create copies of database, Application can talk to either database
• Sharding - Sharding choosing a partition key, Key-value store partition based on key
• Caching - Precomputed and stored, Manage cache expiration time and refresh logic

For streaming data we had already discussed Events Hub, Apache kafka. Now we have something called KSQL (Kafka streaming SQL to run on continuous data)

Great Session Talk

 

RDBMS VS NOSQL Considerations, Quick Summary

• Performance - Latency tolerance
• Durability - Data loss tolerance
• Consistency - Weird results tolerance (Dirty data tolerance)
• Availability - Downtime tolerance

Happy Learning!!!