Open Source Test Tools Vs Commercial Test Tools

I have never had the taste of working with record - playback tools. I have mostly developed custom tools / scripts for qa tasks / deployment tasks. I work in different streams Database DEV - QA - Tools Development - Performance & Big Data too. Working on different areas provides a different perspective than doing repetitive things. My perspective of QA evolves with reusable scripts for data generation / scripts that simplify / eliminate repetitive tasks during deployment / configuration / testing / validating.

I have worked with Selenium, Coded UI, Custom developed Automation test frameworks. I have observed new engineering efforts for Automation test framework in every company I worked for. Either the code base becomes too big to manage / modify or new folks hired move towards develop from scratch than maintenance efforts. It is questionable ROI calculation. When the quality of DEV is poor every small bug in QA might show up as hundreds of bugs. How much of this bugs can actually be identified by basic QA check by DEV is another point to consider measuring QA bugs.

QA efforts are often viewed as commodity efforts where focus is mainly to deliver and repetitive cycles of testing are acceptable. Instead of such a model joint DEV-QA effort would always help to identify most bugs before releasing it to QA.

Both Open Source Tools Vs Commerical tools helps address test automation challenges. One instance. Working with WINCE apps, Its very difficult to automate hardware - software integration workflows. Testcomplete tool eliminated most of this efforts with emulating the actions on Mymobiler which in turn mimics real user on WINCE installed hardware.

Effort involved to automated WINCE App deployed on a device vs using a tool like test complete to completely eliminate the pain-point in writing WIN32 calls (Send message, Send Keys) is worth evaluating before thinking of license cost.

Overall its a mix of Open Source Tools, Commercial Tools, In-House scripts, Quality practices in coding, unit testing only can ensure a Quality product. Responsibilities are not just for one function but every function need to be accountable / responsible to deliver a Quality Product.

Automation tools are primarily viewed as Record / Playback, Automation Framework Implementation. More than it they can be also leveraged for

• Throw Away Scripts to Aid Functional Testers to eliminate repetitive tasks
• Automation tool can be used to support system activities during functional testing - Monitoring, Screen Capture, Aid Testing by Simulating user events during tests / Support long running tests  
• Extend it for Support / UAT Environments for for Deployment / Installation where installation involves several client / server / web components installation
• Aid Automated Deployments / Un-Installations

Happy Learning!!!

Multithreading - Automation Basics - Usage of lock to ensure threadsafe

Example Code

• Usage of lock to ensure threadsafe

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.Threading;

namespace MultithreadedApp

{

    class Program

    {

        //static long _value1;

        private object threadLock = new object();

        void runcode()

        {

            lock (threadLock)

            {

                for (int i = 0; i < 100; i++)

                {

                    Console.WriteLine("Value of _value1 " + i);

                }

            }

        }

        static void Main(string[] args)

        {

            Thread[] agents = new Thread[10];

            Program[] P = new Program[10];

            for (int i = 0; i < 10; i++)

            {

                P[i] = new Program();

                agents[i] = new Thread(P[i].runcode);

                agents[i].Start();

            }

            Console.ReadLine();

        }

    }

}

• Thread Safe Lock, Volatile and Interlock in C#
• MULTI-THREADING III - CONCURRENCY 2014
• Run your Unit Tests in Parallel
• ADVANCED THREADING
• C# Interlocked
• C# Multithreading Example
• Java Thread Pool Example using Executors and ThreadPoolExecutor
• Java concurrency (multi-threading) - Tutorial

Happy Learning!!!

Regression Test Automation Myths

When do we need to automate test cases ?

Large amount of test effort would be spent for Regression testing for any Change Request (CR#) received. The Amount of effort spent for Automating Regression Test Cases Vs Time saved per Regression Test Cycle is required before we automate.

Example: Effort Required for Regression Test Automation - 200 Hrs which includes Framework setup, Test Data creation, Test Execution layer and Reporting of test results. If the Features might change in next subsequent release the effort required to maintain the Automation suite also need to be considered.

Automation of test cases not necessarily result in savings of test effort. It also depends on maintainbility of the regression suite.
For Automating Regression Test cases couple of guidelines listed below.
1. Managing Input test data

• Input can be stored from a table, Expected output can also be stored in a table.
• Input can be obtained by running a SQL Query

2. Execution the test - Execute Function/Web Services/Method
3. Obtain the output

• Parse the output fetch the result value
• Query the Database to verify Data is present
• Compare with result data set and verify expected result is obtained result

4. Consolidating and Reporting findings.

A robust test automation framework is expected to execute all the test cases report the results with all details like %% of Test cases passed, test cases failed, Errors Reported.

More Reads
Agile-Friendly Test Automation Tools/Frameworks

Another Good Read

Automated Testing vs Manual Testing

View more presentations from Directi Group.

State: published Open Source Automated Test Tools Written in Java

Test Automation Best Practices (high-level presentation)

View more presentations from Leonard Fingerman.

Test Automation Strategies For Agile

View more presentations from Naresh Jain.