Value Realization of Test Function / Accessing Maturity of QA Organization

Value Realization of Test Function

• Ensure timely launch of products
• Quality Focus, Meeting or beating the Schedule
• Early Involvement of Test during Design, Requirements Phase
• Adopt Mix of White Box + Black Box Testing. Learn the internals/Design of the product.
• Cover all aspects during Testing (Security, Performance, Error Handling, and Usability)
• Accountability and own Deliverable when things are driven by minimal documentation.
• Proactive in highlighting issues-risks during execution
• Automation of Frequent tasks (Regression Testing, Build Verification Test Cases)
• Analyze and Evaluate new approaches to minimize test effort. Prototype and evaluate small improvements
• Learn the Product to own the product and Test the Product
• Add enough documentation on features, Ramp-up documents, Video recording sessions of product walkthru. Learn the product and share your learning's with the Team 

Followup for this blog post, wrote another post for Accessing Maturity of QA Organization


In this post we will look at aspects in evaluating QA organization. These parameters would help us access maturity of team / growth aspects / learning opportunities for the team.

Guidelines provided are based on my experience working with different test teams.

1. Test Process

• At what stage of project execution test team gets involved
• Test methodology V-model, SCRUM approach (test early, multiple build cycles, feature set delivered incremental basis)
• Delieverables for test team during various stages of project execution
• How test coverage is achieved
• How test estimations are arrived ?
• What metrics are captured for test execution, test scenario coverage, production bugs

2. Types of Testing Done

• What are the different types of testing done - Web Services, UI, Database, BI, Sharepoint, Biztalk etc..
• This parameter would help us determine ownership of test team. Complete product testing is owned by a team or distributed ownership

3. Performance, Scalability and Security Aspects

• What tools / technologies are used for evaluating performace aspects of the application
• What tools are used to validate security aspects of the application
• Inhouse tools / commercial tools available in market ?
• Application testing involves functional, performance and security testing. This parameter would help us understand tool used, testing methodology adopted, skill sets/ areas of expertise of team.

4. Automation Aspects

• What is the percentage of automation. Is it based on commercial tools / Inhouse tools developed ?
• Is it maintainable automation. Does it have a continious integration setup ? Is Automation used by development team ?
• This parameter would help us understand the development methodology. Is it TDD model ?. Automation should not be done for the sake of it. There should be measurable ROI
• I am very much convinced with the 3 layer approach in blog post
• GUI Level, Methods Level and Database Checks. KISS ("Keep it simple, Stupid!")

5. Release Cycles

• What is the frequency of Release Cycle. What is the test strategy for it. Release cycle would help us understand the development methodology SCRUM or waterfall or any other approach?

6. Roadmap for Team

Team should have a roadmap to define

• Where do we stand at this point in terms of Quality
• What are our goals in next 3~6 months
• Clear roadmap to achieve this in terms of
• Initiatives
• Technical Trainings
• Tools Adoption
• Process Changes
• Team accomplishments for previous year (last 3~6 months roadmap)

7. Metrics

I am tired of this parameter. I have witnessed at times Zero Bugs as target. If there are Zero bugs in software it means bugs have not been found out. Unless we prove

• 100% code Coverage in terms of test case execution
• 100% P1/P2 Cases passed

These two parameters are essential. Given the current market scenario. Test Approach is predominantly Black Box and Grey Box(Black Box with a little White Box) approach.

There might be few more additions in terms of different testing domains (Banking, Telecom, Ecommerce space etc..)

Product testing would involve extensive White box testing, Developing Test Automation Frameworks for performance, code coverage & functional testing.

These are high level parameters to get started. Further analysis on above listed areas would provide more clarity in execution model.

Related Reads - Agile!! Agile!! Agile!! Will it be Fragile!!
Is the “Joel Test” appropriate when looking for a QA job?
Build your own model of software testing – or rediscover one from several thousand years ago
Choosing and Managing the Ideal Test Team

Happy Reading!!