Value Realization of Test Function / Accessing Maturity of QA Organization

Value Realization of Test Function

• Ensure timely launch of products
• Quality Focus, Meeting or beating the Schedule
• Early Involvement of Test during Design, Requirements Phase
• Adopt Mix of White Box + Black Box Testing. Learn the internals/Design of the product.
• Cover all aspects during Testing (Security, Performance, Error Handling, and Usability)
• Accountability and own Deliverable when things are driven by minimal documentation.
• Proactive in highlighting issues-risks during execution
• Automation of Frequent tasks (Regression Testing, Build Verification Test Cases)
• Analyze and Evaluate new approaches to minimize test effort. Prototype and evaluate small improvements
• Learn the Product to own the product and Test the Product
• Add enough documentation on features, Ramp-up documents, Video recording sessions of product walkthru. Learn the product and share your learning's with the Team 

Followup for this blog post, wrote another post for Accessing Maturity of QA Organization


In this post we will look at aspects in evaluating QA organization. These parameters would help us access maturity of team / growth aspects / learning opportunities for the team.

Guidelines provided are based on my experience working with different test teams.

1. Test Process

• At what stage of project execution test team gets involved
• Test methodology V-model, SCRUM approach (test early, multiple build cycles, feature set delivered incremental basis)
• Delieverables for test team during various stages of project execution
• How test coverage is achieved
• How test estimations are arrived ?
• What metrics are captured for test execution, test scenario coverage, production bugs

2. Types of Testing Done

• What are the different types of testing done - Web Services, UI, Database, BI, Sharepoint, Biztalk etc..
• This parameter would help us determine ownership of test team. Complete product testing is owned by a team or distributed ownership

3. Performance, Scalability and Security Aspects

• What tools / technologies are used for evaluating performace aspects of the application
• What tools are used to validate security aspects of the application
• Inhouse tools / commercial tools available in market ?
• Application testing involves functional, performance and security testing. This parameter would help us understand tool used, testing methodology adopted, skill sets/ areas of expertise of team.

4. Automation Aspects

• What is the percentage of automation. Is it based on commercial tools / Inhouse tools developed ?
• Is it maintainable automation. Does it have a continious integration setup ? Is Automation used by development team ?
• This parameter would help us understand the development methodology. Is it TDD model ?. Automation should not be done for the sake of it. There should be measurable ROI
• I am very much convinced with the 3 layer approach in blog post
• GUI Level, Methods Level and Database Checks. KISS ("Keep it simple, Stupid!")

5. Release Cycles

• What is the frequency of Release Cycle. What is the test strategy for it. Release cycle would help us understand the development methodology SCRUM or waterfall or any other approach?

6. Roadmap for Team

Team should have a roadmap to define

• Where do we stand at this point in terms of Quality
• What are our goals in next 3~6 months
• Clear roadmap to achieve this in terms of
• Initiatives
• Technical Trainings
• Tools Adoption
• Process Changes
• Team accomplishments for previous year (last 3~6 months roadmap)

7. Metrics

I am tired of this parameter. I have witnessed at times Zero Bugs as target. If there are Zero bugs in software it means bugs have not been found out. Unless we prove

• 100% code Coverage in terms of test case execution
• 100% P1/P2 Cases passed

These two parameters are essential. Given the current market scenario. Test Approach is predominantly Black Box and Grey Box(Black Box with a little White Box) approach.

There might be few more additions in terms of different testing domains (Banking, Telecom, Ecommerce space etc..)

Product testing would involve extensive White box testing, Developing Test Automation Frameworks for performance, code coverage & functional testing.

These are high level parameters to get started. Further analysis on above listed areas would provide more clarity in execution model.

Related Reads - Agile!! Agile!! Agile!! Will it be Fragile!!
Is the “Joel Test” appropriate when looking for a QA job?
Build your own model of software testing – or rediscover one from several thousand years ago
Choosing and Managing the Ideal Test Team

Happy Reading!!

Inspiring Entrepreneurs

Evan Williams - Creator of blogger and twitter
Matthew Charles Mullenweg - Wordpress founder
Very Good Article - Startup Ideas We'd Like to Fund

All this good info I got during Coffee Chat with my friend Bala. Thanks to him for sharing the info.

Happy Reading!!

Dev Task Estimations, Test Task Estimates

Estimations

I am still learning to provide accurate estimates @ work. Providing estimates can be more accurate when you are familiar with the application, architecture and technology. Given the focus is on SCRUM, Frequent deployments. Arriving at accurate estimate is an ongoing learning.

Being a Developer  - Learning's

• Provide X hours as Estimates thinking about coding, design documents and testing efforts
• Provided Y task time for unit testing efforts, Design Review documents would be less compared to Bug Fixes in Unit test cycle. Finally  you would have spent more than estimated time
• In the eleventh hour before release a critical scenario fails, resulting in late night work on the day before/of release, It is tough to test everything when you develop it :). Your perception is always based on how you have coded it.
• You provide an estimate based on particular approach, when the design you may identify a technical limitation/ blocker in coding/design phase
• There are cases where initial estimate is - 20% more than actually arrived effort. When you find solutions quickly, you end up having a comfortable release schedule
• Estimation accuracy depends on technology, application familiarity, and change complexity
• Practice-Practice-Nail down at every sub-task and have clarity @ granular level
• Doing testing in Dev Environment also will help in identifying bugs @ early stage, This can be no-subsitute to QA effort instead add-on effort.

Being a Tester - Learning's

• Any amount of testing is not sufficient; always there is a little suspicion. Did I check this feature
• Identifying all bugs in first round of testing is important
• If testing stops after finding few bugs then we may not identify all the bugs. Testing should not stop based on bugs identified
• Bug fix cycles and bugs introduced by bug fixes are critical to determine code quality
• Logging Actual Hours – Often you spend more time than estimated time, This has been the case for me
• A proper planning – estimated rough plan would be good to track. I have seen actual planning adding 20-30% more time but without a detailed plan it would be tough to prioritize and execute tasks
• Never commit to a timeline and work backwards, Plan for every task before you commit for schedule

Ex-

• 2-2.30 –Verify X feature
• 2.30-3 – Execute Test cases 1-5
• 3-3.30- Attend Bug Triage

Things when organized appear more simplified than ad-hoc work schedule

Approaching Test Estimation Challenges

• Understanding the Application
•  Integration Layer - Third party or External Apps that communicate with this App
•  Security Layer - Security Mechanism implemented in the App
•  Criticality of the Application - Business Impact when this App is down. Time Vs Impact
• Communication with Business, Developing Effective Acceptance Cases. Think from a Customer Perspective
• Knowledge of Domain & Technology - Previous Experience on projects worked
• Interaction with Expert Testers - Domain QA Experts, Involve-Interact and get their feedback on Scenarios, Test Plans
• Be inquistive, think of areas where it can fail, Review meeting to get feedback from all Project members
• Learn from past mistakes and try not to repeat them

More Reads

Thoughts on Testing - Breaking Down the Test Plan
How Can I Estimate My Testing?

Happy Reading!!

Lessons Learnt - Application Support Project Transition

After transitioning Test, Support Projects in my previous roles. Below listed are activities involved for a project transition. From the management perspective when the project is signed off, statement of work is defined. Initial team setup is the important period.

Application Support - Project Transition

Nature of work for Application Support

• Deploying build in environments (Test, UAT, Production)
• Production Outage management
• Production Monitoring
• Incident and Change Management
• Coordination with other team to resolve DB, Network issues (Other than Application Issues)
• Perigrine is one Incident management tool that I worked with

Identifying Resources 

• Identifying resources for the projects (Entry level, Mid, Senior)
• Identifying managers for the projects
• Hiring –Team setup and hiring involving the client and managers

Typically KT (Knowledge Transfer) involves 

• Walk thru Sessions
• Understand the support model
• Understand the Application, Technology Architecture
• Onsite Travel for senior resources
• Reviewing and understanding documentation of current application
• Shadowing of resources, Client does the work, On-boarding team will shadow (Primary Shadowing)
• Publish learning and understanding documents
• Milestones would be set to cover the related applications (Walkthru, Demo, Deployment in Dev Environments)
• Secondary Shadow. Client monitors when we do the actual work

Ramp-up of Team

• Typically team would be involved supporting Severity2, Severity3 issues to start with
• In a defined time frame 1-2-3 months the team would be ramped up to handle the complete application 

Metrics Measured
In all phases of transition and post transition metrics play a key role with respect to revenues :)

• SLA is associated with tickets of each priority. Number of tickets successfully resolved within SLA
• Number of tickets successfully closed Vs Number of tickets received in queue
• Production Deployments handled

This is based on my experience on transitioning application support projects. There are other additional areas for Network Monitoring, DBA transitioning teams which is not covered in this blog post.

Happy Reading!!!

Developer Vs Tester Vs Program manager Vs Support Analyst

I remember discussions on particular function (Dev, Test, Support, Program management) is good.

• Testers aspiring to become a Developer
• Support Analyst moving into Tester roles
• Tester moving into Program Management roles

Which function is good and easy to work with ? I remember gladiator movie motivational talk "Those who like what they do, They are always proud of doing it". Always, the other side looks green. You will experience unique distinct learnings in every function.

Program Manager

• Understand business
• Convert Business Requirements into Clear, detailed Functional Spec
• Meet the schedule with allocated cost and Schedule
• Meet the deadline or beat the deadline
• Explain and cover all possible use cases for intended functionality

Architect

• Architect the System
• System is Robust, Scaleable
• Follow defined guidelines, ex-SOA Guidelines, DB Guidelines
• Plan for Technology Adoption and Migration for current application to Latest Platforms

Developer

• Follow defined Coding guidelines in developing the System
• Design, Develop and Deliver Design Documents
• Provide Supportability and Monitoring in the System
• Cover all Unit Test Cases and Basic Integration Scenarios
• Development function is the primary impacted function in adopting a SDLC model (Waterfall, Scrum, Peer programming)
• This mindshift change/ Frequent builds depends on the model adopted
• Keep track of upcoming technologies/trends and evaluate it

Tester

• Gatekeepers of Quality
• Verify the Design
• Validate the Functionality
• Test Plan, Test Execution, Bugs, Verifying Fixes
• Test Supportability and Monitoring aspects of the Application
• Performance Testing to benchmark the system based on provided hardware configuration
• Test Automation

Support Analyst

• Production Monitoring
• Deployment, Downtime management
• Managing Enduser queries
• Hardware Migration/Upgrade

Every Job has its own learning. I have worked for Support, Dev and Test Functions. All 3 roles I had good learnings. Its important to understand and appreciate the work delivered by each function. "What you hear, you forget; what you see, you remember; what you do, you understand.. - Unknown Author"
Note - DBA Role, Business Analyst, Onsite Coordinator, Project Management in IT Service Companies I don't have any exposure to it......

Lessons Learned About Being a Software Architect By Trying to Be One
Leveling Up & Leveling Out: Assessing Your Team's Skills


More Reads
Training Your Manager
Business-focused vs. People-focused managers
Guidelines for writing good individual review comments


Happy Reading!!

Web Crawler

• Web crawler is a program that accesses a web site and traverses through the site by following the links present on the pages.
• Whenever we search for a keyword and go to a site.
• Google engine collects information on the site clicked by the user using its crawler tool.
• Googlebot is Google’s web crawling tool, which finds and retrieves pages on the web and hands them off to the Google indexer.

Good Reads
Crawling a web sites with HtmlAgilityPack

Experiences with the WebCrawler Anatomy of a Search Engine 
How Google Works
Web crawler architectures
Google vs. Bing: Correlation Analysis of Ranking Elements

Bing vs. Google: Prominence of Ranking Elements
How To: Detect Web Crawlers, Spiders, & Robots in ASP.NET
How to detect search engine crawlers?
Web Crawler
How Search Engines Work
Distributed web crawling
History of Search: Search Engine Timeline