Weekend Reading - Webinar - Performance Testing Approach for Big Data Applications.

Very good session – Webinar - Performance Testing Approach for Big Data Applications. Few interesting notes / slides from session

• Rate of Data Ingestion - How fast system consumes data?
• Data Processing - Speed how data is processed. Testing Data processing in isolation with data sets populated. Run specific perf tests (MR Jobs, Pig, Hive Scripts)
• Data Persistence – I/O bound process. (Data Writes / Updates on DB, Garbage Collection, Monitoring Metrics)
• Complete end to end time for processing (Network Connectivity, Processing, Results)

Big Data Test Challenges

• Diverse Technologies
• Unavailability of Test Tools for Big Data Technologies / Scenarios
• Limited Monitoring / Diagnostic Solutions
• Test Scripting / Environment

Perf Test Tools

• Use cloud to simulate large infrastructure
• Cloud orchestration scripts Puppet, Chef

Approach

• Depending on usage in production identify patterns for production workload
• Fault Tolerance Scenarios
• Hadoop monitoring tools to check Map reduce jobs
• Selecting Test Clients - Custom code
• Performance / Failover tests to ensure scalability (Node failures during processing)

Test Parameters and Summary

Very Nice, Practical and useful webinar. There are a lot of posts / webinars. This one is very useful and practical.

More Reads
Evaluating SolrMeter for Performance Testing
Benchmarking with HTTPerf.js and NodeUnit

Happy Learning!!!