"No one is harder on a talented person than the person themselves" - Linda Wilkinson ; "Trust your guts and don't follow the herd" ; "Validate direction not destination" ;

March 31, 2011

Frequently Asked Question to QA Team

When Dev Team Takes 1 Day for Development, Why Does Test need 2 Days for Testing
  • Development involves Design, Coding, Unit Testing, Release
  • Quality of Development Deliverables can be measured by Bugs found in Test
  • Quality of Test Deliverables can be measured by Bugs found in Production
  •  With Pure black box testing, possibilities of reducing test estimates may prove risky
  • With a mix of black box and white box testing, You can take a calculated risk of Areas of Scope (Must be covered) Vs Sanity Testing in other Areas
  • Estimates should be looked at what are the areas covered in testing, Comparision of Development and Test Estimates is *NOT a* valid argument
1 Line of Code Change, Why Do you need 1 Day for Testing ?
Most common question for Test Team. What it takes to Deliver it by XX-XX-XXXX Date ?
Can we cut down Test Estimates by 1 Day ?
  • Estimates should be looked to analyze Tasks Covered, Scope of Testing, Percentage of Test Automation in place. 
  • With a mix of black box and white box testing, You can take a calculated risk of Areas of Scope (Must be covered) Vs Sanity Testing in other Areas
Why Automation did not catch this bug ?
  • Domain expertise comes with experience and exposure. Good Testers have very good domain knowledge of product
  • Obvious bugs# can be caught by automation. Layout related / Integration issues would be caught during manual exploratory testing
  • Automation plus manual exploratory testing would be good approach to maintain high Quality product
We need 100% Automation
  • Building automation framework before product lauch and Priority 1 cases Automation before product release is a good approach dealing with automation
  • Based on events captured in production, User Workflows need to converted into equivalent integration scenarios to simulate end user scenarios
  • From the production bugs# keep enhancing the Automation suite
Happy Reading!!

March 24, 2011

Selenium - Demo Test Page










Selenium 2 - Good Reads

Good Read -
Exploring Selenium 2: Introduction & Lessons Learned in Selenium 1 - Selenium RC is no more required in Selenium2, You can directly manipulate the browser.

Posting below presentation after confirming with Jason Huggins.
Selenium 2 Webinar: The Next Generation of Web and Mobile Application Testing




Hope videos are posted soon in site
Selenium 2.0 beta 1 Release
Selenium 2 in .NET framework
What is Selenium2
Web Driver Roadmap
Selenium 2.0 and web driver
Migrating to Selenium 2
It's time for QTP to Die

Happy Reading!!

March 18, 2011

TestNG Example - testng - Passing parmeters in XML File

Our next post is based on my question in stackoverflow site. Let's try an example based on Cedric reply.

Below is example code
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import com.thoughtworks.selenium.Selenium;
import static org.testng.Assert.assertTrue;
import org.testng.annotations.*;
import com.thoughtworks.selenium.*;
public class TestNGXMLData
{
@Test
@Parameters(value={"A","B","C","D"})
public void xmlDataTest(String A, String B, String C, String D)
{ 
            System.out.println("Passing Three parameter to Test "+A +" and "+B + " and "+C);
            String[] elements = D.split(",");
            for (String element : elements)
            {
                  System.out.println("Item is "+element);
                 
            }
}
}

XML for setting parameters

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite" parallel="none">
  <test name="Test" preserve-order="false">
   <parameter name="A" value="1"/> 
   <parameter name="B" value="2"/> 
   <parameter name="C" value="3"/> 
   <parameter name="D" value="4,5,6,7,8"/> 
    <classes>
      <class name="TestNGXMLData"/>
    </classes>
  </test>
</suite>

Right Click on XML and run it as TestNG Test. Below is output in console window
Passing Three parameter to Test 1 and 2 and 3
Item is 4
Item is 5
Item is 6
Item is 7
Item is 8

Extending this example and adding multiple tests and setting it for parallel execution and preserve test order.

Modified XML File
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite" parallel="classes" thread-count="5">
  <test name="firstTest" preserve-order="true">
   <parameter name="A" value="1"/> 
   <parameter name="B" value="2"/> 
   <parameter name="C" value="3"/> 
   <parameter name="D" value="4,5,6,7,8"/> 
    <classes>
      <class name="TestNGXMLData"/>
    </classes>
  </test>
  <test name="secondTest" preserve-order="true">
   <parameter name="A" value="10"/> 
   <parameter name="B" value="20"/> 
   <parameter name="C" value="30"/> 
   <parameter name="D" value="40,50,60,70,80"/> 
    <classes>
      <class name="TestNGXMLData"/>
    </classes>
  </test>
  <test name="thirdTest" preserve-order="true">
   <parameter name="A" value="100"/> 
   <parameter name="B" value="200"/> 
   <parameter name="C" value="300"/> 
   <parameter name="D" value="400,500,600,700,800"/> 
    <classes>
      <class name="TestNGXMLData"/>
    </classes>    
  </test>
</suite>

Modified code adding try-catch block (Keeping up with Coding Guidelines :))

import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import com.thoughtworks.selenium.Selenium;
import static org.testng.Assert.assertTrue;
import org.testng.annotations.*;
import com.thoughtworks.selenium.*;
public class TestNGXMLData
{
@Test
@Parameters(value={"A","B","C","D"})
public void xmlDataTest(String A, String B, String C, String D)
{ 
            try
            {
                  System.out.println("Passing Three parameter to Test "+A +" and "+B + " and "+C);
                  String[] elements = D.split(",");
                  for (String element : elements)
                  {
                        System.out.println("Item is "+element);
                       
                  }
            }
            catch(Exception e)
            {
                  System.out.println("Exception Occurred "+ e.toString());
            }
}
}

Output

Passing Three parameter to Test 1 and 2 and 3
Item is 4
Item is 5
Item is 6
Item is 7
Item is 8
Passing Three parameter to Test 10 and 20 and 30
Item is 40
Item is 50
Item is 60
Item is 70
Item is 80
Passing Three parameter to Test 100 and 200 and 300
Item is 400
Item is 500
Item is 600
Item is 700
Item is 800

Happy Reading!!

March 13, 2011

QA Skills

My Summary of QA skills for Junior/Senior QA Candidate

Junior QA
  • Follows defined QA Process
  • Prepares Bugs Reports, Status Reporting with all relevant information
  • Manages and updates automation suite
Senior QA
  • Self Organized
  • Challenges the Environment, Raise testing standards by experimenting with new processes/Adopting new Tools
  • Aware of Test Automation Framework Design and Development 
  • Sound Technical Skills 
  • Selfless - Volunteers/mentors team in terms of process/sharing best practices
  • Actively involves/drives meaningful discussions and implements best practices  
  • Takes calculated risks and delivers products meeting quality bar
Growing to a Test Architect Role
Impressive QA JD description - link


Every skill (DEV / QA / Automation) can be learned with interest and passion. This JD is a mix of DEV-QA-Automation. This is a very good write up.


One more interesting job-ad from link. Highlighted below are core skills / competencies. This co-relates to a lot of behavior traits



More Reads

March 04, 2011

Tools to evaluate for Web Testing

Time is limited, Delivering new products to customers plays a key role. Efficiency in terms of cost, time and effort is the key.

To finalize a test tool for functional/perf test automation below are few guidelines/considerations
  • Pick Top 3 test tools in terms of Features provided, Cost, Cost of Training
  • Review the tools using Trial versions/Demos
  • Analyse effort required to learn the tools, develop the framework and manage the framework
  • Evaluate cost for developing in-house tool vs Using Open Source Tools vs Licensing a Tool
Couple of Interesting tools to try out are

 More Reads

Happy Reading!!

February 18, 2011

Test Automation Framework Using Selenium - Part III

[Next Post in Series]


In continuation with previous posts we are now going to look at logging. Please refer to TestNG earlier posts for examples. I attribute my learning on logging to post. Thanks to Bindu Laxminarayan.

Added two files SuiteReporter.java and TestReporter.java. You can copy/paste the contents of file from the blog post. You can right click on TestRun.java and select option Convert to TestNG as per below snapshot

We need to edit the xml file as provided below testng.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite" parallel="none">
      <listeners>
   <listener class-name="SuiteReporter" />
   <listener class-name="TestReporter" />
   </listeners>
   <test name="Test" preserve-order="false">
    <classes>
      <class name="TestRun"/>
    </classes>
  </test>
</suite>

You can run TestRun.java as TestNG test. In the console window you will see below entries logged


After running TestNG test you would also see emailable-report.html. Please find snapshot



Improvements (Next Upcoming Posts)
  • Continous Integration Server
Pending Learning are below topics
  • Watir, Sahi
  • JavaScript, Python, Ruby
We have successfully covered basics of automation framework design, coding standards, data driven testing, reports and Logging as well. You can install Eclipse TestNG plug in from link

More Reads -
Better reporting with ReportNG
Using Firebug for Load Testing
TestNG XML Driven Parameters
Using Firebug in IE
How to setup a Test Automation Infrastructure using Selenium, JUnit, Hudson & ANT

Happy Reading!!

January 30, 2011

Test Automation Framework Using Selenium - Part II

[Next Post in Series]

In Continuation with Previous post We will be looking at Adding Data Driven Tests using TestNG. Please refer to TestNG earlier posts for examples

TestRun.Java - In this file we instantiate selenium instance, open the page. Data Providers is set in this file.
FirstPage.Java - This has methods for opening first page and verifying first page items. Assert statements are added to verify. Removed print statements that was present in earlier post.
SecondPage.Java - Seperating methods and verifiers for that particular page to keep test suite maintainable. Three verifiers are created for ordered list, unordered list and verify links.

Improvements (Next Upcoming Posts)
  • Logging
  • Reporting
  • Continous Integration Server
In next posts I plan to learn and Implement improvements.
1. Create a project and add required libraries. Verify earlier post
2. TestRun.java


import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import com.thoughtworks.selenium.Selenium;
import static org.testng.Assert.assertTrue;
import org.testng.annotations.*;

import com.thoughtworks.selenium.*;
public class TestRun
{
      Selenium selenium;
      FirstPage fp;
      SecondPage sp;

@BeforeTest
public void setUp()
{
      selenium = new DefaultSelenium("localhost",4444,"*iehta","http://sqlandsiva.blogspot.com/");
      selenium.start();
}

@AfterTest
public void tearDown()
{
      selenium.stop();
}

@Test(dataProvider = "firstPageList")
public void firstPageTestCase( ArrayList<String> listFirstPageExpectedResult)
{
      fp = new FirstPage(selenium);
      fp.clickFirstPage();
      fp.verifyListItems(listFirstPageExpectedResult);
      sp = fp.openSecondPage();
}

@Test(dataProvider = "unOrderedList")
public void secondPageVerifyUnOrderList(ArrayList<String> unOrderedlistExpectedResult)
{
      sp.verifyUnorderedListItems(unOrderedlistExpectedResult);
}

@Test(dataProvider = "verifyLinks")
public void secondPageVerifyLinks(ArrayList<String> linksExpectedResult)
{
      sp.verifySecondPageLinks(linksExpectedResult);
}

@Test(dataProvider = "orderedList")
public void secondPageVerifyOrderList(ArrayList<String> listExpectedResult)
{
      sp.verifyorderedListItems(listExpectedResult);
}

@DataProvider(name = "unOrderedList")
public Object[][] setDataforUnOrderedList() {

      List<String> listExpectedResult = new ArrayList<String>();
      listExpectedResult.add("Unordered List One");
      listExpectedResult.add("Unordered List Two");
      listExpectedResult.add("Unordered List Three");

      Object[][] retkeyword = {{listExpectedResult}};
      return (retkeyword);
}

@DataProvider(name = "verifyLinks")
public Object[][] setDataforLinks() {

      List<String> listExpectedResult = new ArrayList<String>();
      listExpectedResult.add("Visit sqlandsiva!");
      listExpectedResult.add("Visit brightlife4all!");

      Object[][] retkeyword = {{listExpectedResult}};
      return (retkeyword);
}

@DataProvider(name = "orderedList")
public Object[][] setDataforOrderedList() {

      List<String> listExpectedResult = new ArrayList<String>();
      listExpectedResult.add("Order List One");
      listExpectedResult.add("Order List Two");
      listExpectedResult.add("Order List Three");

      Object[][] retkeyword = {{listExpectedResult}};
      return (retkeyword);
}

@DataProvider(name = "firstPageList")
public Object[][] setDataforFirstPageList()
{
      List<String> listExpectedResult = new ArrayList<String>();
      listExpectedResult.add("a1");
      listExpectedResult.add("a2");
      listExpectedResult.add("a3");
      listExpectedResult.add("b1");
      listExpectedResult.add("b2");
      listExpectedResult.add("b3");
      listExpectedResult.add("c1");
      listExpectedResult.add("c2");
      listExpectedResult.add("c3");

      Object[][] retkeyword = {{listExpectedResult}};
      return (retkeyword);
}
}
3. FirstPage.java
import com.thoughtworks.selenium.Selenium;
import static org.testng.Assert.assertTrue;

import java.util.ArrayList;
import java.util.Vector;
import org.testng.Assert;
import org.testng.annotations.*;
import com.thoughtworks.selenium.*;
public class FirstPage{
Selenium selenium;
public static String FIRST_PAGE_LINK = "/2011/01/test-xml-page-for-selenium.html";
public static String SECOND_PAGE_LINK = "/2011/01/test-page-for-selenium-example.html";

public FirstPage(Selenium selenium)
{
      this.selenium = selenium;
     
}
public FirstPage clickFirstPage()
{
      System.out.println("Opening First Page");
      clickFirstPage(FIRST_PAGE_LINK);
      selenium.waitForPageToLoad("30000");
      return new FirstPage(selenium);
}

public SecondPage openSecondPage()
{
      System.out.println("Opening Second Page");
      clickSecondPage(SECOND_PAGE_LINK);
      selenium.waitForPageToLoad("30000");
      return new SecondPage(selenium);
}

private void clickSecondPage(String number)
{
      selenium.open(SECOND_PAGE_LINK);
}

private void clickFirstPage(String number)
{
      selenium.open(FIRST_PAGE_LINK);
}

public void verifyListItems(ArrayList<String> OrderList)
{
      System.out.println("In FirstPage Test Method");
      try
      {
            System.out.println("Running FirstPage Test");
            int i=1,j=1;
            while(selenium.isElementPresent("//table[@id='firstTable']//tr["+i+"]/td["+j+"]"))
            {
            while(selenium.isElementPresent("//table[@id='firstTable']//tr["+i+"]/td["+j+"]"))
            {
                  Assert.assertTrue(OrderList.contains(selenium.getText("//table[@id='firstTable']//tr["+i+"]/td["+j+"]")));
                  j++;
            }
            j=1;
            i++;
      }
      }
      catch(Exception ex)
      {
                        ex.toString();
      }
}
}
4. SecondPage.Java
import com.thoughtworks.selenium.Selenium;
import static org.testng.Assert.assertTrue;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Vector;
import org.testng.Assert;
import org.testng.annotations.*;

import com.thoughtworks.selenium.*;
public class SecondPage{
Selenium selenium;

public SecondPage(Selenium selenium)
{
      this.selenium = selenium;
}

public SecondPage clickFirstPage()
{
      return new SecondPage(selenium);
}

public void verifyUnorderedListItems(ArrayList<String> OrderList)
{
      System.out.println("In SecondPage UnOrderedList Verifier Method");
      try
      {
            int j=1;
            while(selenium.isElementPresent("//div[@id='UnOrderedList']/ul/li["+j+"]"))
            {
                  Assert.assertTrue(OrderList.contains(selenium.getText("//div[@id='UnOrderedList']/ul/li["+j+"]")));
                  j++;
            }
      }
      catch(Exception ex)
      {
                        ex.toString();
      }
}

public void verifyorderedListItems(ArrayList<String> OrderList)
{
      System.out.println("In SecondPage Ordered List Verfier Test Method");
      try
      {
            int j=1;
            while(selenium.isElementPresent("//div[@id='OrderList']/ol/li["+j+"]"))
            {
                  Assert.assertTrue(OrderList.contains(selenium.getText("//div[@id='OrderList']/ol/li["+j+"]")));
                  j++;
            }
      }
      catch(Exception ex)
      {
                        ex.toString();
      }
}

public void verifySecondPageLinks(ArrayList<String> OrderList)
{
      System.out.println("In SecondPage Links Verifier Method");
      try
      {
            int j=1;
            while(selenium.isElementPresent("//div[@id='Links']/a["+j+"]"))
            {
                  Assert.assertTrue(OrderList.contains(selenium.getText("//div[@id='Links']/a["+j+"]")));
                  j++;
            }
           
      }
      catch(Exception ex)
      {
                        ex.toString();
      }
}
}
Start Selenium Server. Run TestRun.java as TestNG test.  This is a working example. Please find output








More Reads
A Selenium CaptureNetworkTraffic Example in Java
Selenium Example – How to Amend the JavaScript of the Web Application Under Test
Automated Web/HTTP Profiler with Selenium-RC and Python
Happy Learning!!