"No one is harder on a talented person than the person themselves" - Linda Wilkinson ; "Trust your guts and don't follow the herd" ; "Validate direction not destination" ;

April 22, 2011

.NET 2.0 Web Services Functional Test Automation

I'm reading book .Net Test automation recipies. If I had got this book 3 years back this could have helped me alot in automation. Never too late. I love this book. Thanks to the author for his great work. I used to use tool WebServiceStudio for Web Services Testing since 2006 (Long time back :))
  • I have attempted creating a simple asp.net webservice
  • Testing the same using Httpwebrequest POST
This is a skeleton only, The next post I'm planning to cover
  • Trying same example for WCF Service
  • Implement Data Driven Testing for this example using Nunit
  • Logging and Reporting
 Step 1 - Create a simple web service














Step 2 - Below sample project would be created, Modify the code for addition of two numbers


[WebMethod]
public int AddNumbers(int a, int b)
{
    return a+b;
}

Step 3 - View in Browser, You will see the SOAP Request



Step 4 - Created a Console Application

Copy - Paste below code in the Console App. Provide the correct modified URL for the webapp.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.Data;
using System.IO;
namespace WebServiceAutomation
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Calling AddSum webmethod");
            int input1 = 10;
            int input2 = 20;
            int expectedresult = 30;
            int actualresult;
            string postData = @"<?xml version=""1.0"" encoding=""utf-8""?>
                <soap:Envelope xmlns:xsi=""http://www.w3.org/2001/XMLSchema-instance"" xmlns:xsd=""http://www.w3.org/2001/XMLSchema"" xmlns:soap=""http://schemas.xmlsoap.org/soap/envelope/"">
                <soap:Body>
                <AddNumbers xmlns=""http://tempuri.org/"">
                <a>input1</a>
                <b>input2</b>
                </AddNumbers>
            </soap:Body>
            </soap:Envelope>";
            postData = postData.Replace("input1",input1.ToString());
            postData = postData.Replace("input2",input2.ToString());
            byte[] buffer = Encoding.ASCII.GetBytes(postData);
            string url = "http://localhost:1033/Service1.asmx?op=AddNumbers";
            HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
            req.Method = "POST";
            req.ContentType =  "text/xml";
            req.ContentLength = buffer.Length;
            req.Timeout = 5000;
            Stream request = req.GetRequestStream();
            request.Write(buffer, 0, buffer.Length);
            request.Flush();
            request.Close();
            HttpWebResponse res = (HttpWebResponse)req.GetResponse();
            Stream resst = res.GetResponseStream();
            StreamReader sr = new StreamReader(resst);
            string response = sr.ReadToEnd();
            Console.WriteLine("HTTP Response is " + response);
            Console.ReadKey();
            int start = response.IndexOf("<AddNumbersResult>");
            int end = response.IndexOf("</AddNumbersResult>");
            string Actualresult = response.Substring(start+18, end - start-18);
            Console.WriteLine("Expectd Result is " + expectedresult +"\n");
            Console.WriteLine("Actual Result is " + Actualresult + "\n");
            Console.ReadKey();
        }
    }
}

Step 5 - Output Window After executing the Console App

Explanation for the code
  • If you want to interact more directly with a Web server, the HttpWebRequest and HttpWebResponse add additional methods that make this easier
  • HttpWebRequest and HttpWebResponse class is inside the System.NET namespace, and this two classes is designed to communicate by using the Http Protocol
  • HttpWebRequest can be used to send HTTP request to the server
  • Stream Class – msdn link  
  • The Stream class is abstract
    Means that same code can be used to read from or write to any kind of secondary storage (aka the backing store)
  • The StreamWriter class is derived from an abstract class called TextWriter, which writes characters to a stream or file
    The StreamReader class is derived from an abstract class called TextReader, which reads characters from a stream or file
  • StreamWriter is quite a bit different than StreamReader
    • StreamReader converts text to string or char[ ]
    • It does not do "text to numeric/object" conversions
    • You have to manually parse and convert the input
  • StreamWriter automatically converts all types of binary data to human-readable text for display
  • Stream Reader
    • Read a line from the file using the ReadLine() method
    • Returns a string if OK, null on end-of-file
    • Stops reading when newline or "\r\n" encountered
    • End-of-line is discarded
    • Read entire file into a single string using ReadToEnd()
More Reads


SOAP Vs REST (Representative State Transfer (REST))
  1. REST stands for Representational State Transfer, this basically means that each unique URL is a representation of some object
  2. It embraces a stateless client-server architecture in which the web services are viewed as resources and can be identified by their URLs. Source - Link
  3. REST has no WSDL interface definition
  4. REST is over HTTP, but SOAP can be over any transport protocols such HTTP, FTP, STMP, JMS etc.
  5. SOAP is using soap envelope, but REST is just XML
  6. SOAP RPC: Verb based. GetUser(string userId), AddUser(User u)
  7. REST: Noun based. Ex- http://mysite/users/dseven, http://mysite/users/newUserForm
  8. REST does NOT require a separate page for each data item, just a separate address.

Good Read - REST vs. SOAP – The Right WebService
Soap Headers Authentication in Web Services
CodeSnip: Handling SOAP Exceptions in Web Services
Interview Question: Compare two web services type SOAP and RESTful (SOAP Vs RESTful)
XML comparison tutorial using XMLUnit
Consuming Twitter APIs

Hope it Helps.
Happy Reading!!

April 02, 2011

Selenium Web Test - Automation Framework - Best Practices

[Next Post for Selenium Based Test Automation Framework]

This post is based my question in stackoverflow site. These questions help us learn best practices while developing an automation framework.

Summarizing Discussion Points
  • Page Object test design pattern is a good approach in building automation framework. Based on it only I have written example posts
  • Don't use the Selenium IDE generated code as Automation Framework base
  • Build a very good underlying framework architecture and refactor it as you keep updating the suite
  • Learn Design patterns and Apply it wherever applicable
  • 3 layer test automation framework is very good. I feel convinced with this approach.
    • GUI Level Testing
    • Methods Level Testing
    • Database Checks
Posting below list of good reads on Test Automation

Integration Test Principles - Notes from the post
  • Configuration should be minimal (Pointing to a different Db to run a test)
  • Test cases should be independent (Select and run few cases)
  • Tests should be efficient
  • The test suite should clean up after itself (Good to clear logs/data)
Please feel free to check my learnings for Developing Test Automation Framework using Selenium.

Any comments / suggestions please comment on this blog post / alternatively you can connect on my email provided in homepage.



Happy Reading!!

Guidelines in Providing Test estimates

Being Agile means managing changing requirements, accommodating frequent releases, having very good QA processes to support the same. Below blog post is on communicating test Estimates. It's all hypothetical scenarios :). I'm sharing my personal opinion. Before you actually test a project you would be asked to provide test estimates for upcoming projects.

Let’s take below reply
Explanation One
  • For testing the changes in this feature, It would require 2 days
Let's see how this explanation can be probed - Peer QA might say, in my opinion, these changes would take only one day to test

Explanation two
  • This changes would require 2 days for testing
  • I would be covering below test scenarios
Let's see how this explanation can be probed - Few scenarios are missed, below scenarios can be additionally covered

Explanation three
  • This changes would require 2 days for testing
  • I would be covering below test scenarios
  • In scope - Below Cases
  • Out of Scope - A and B Features
Let's see how this explanation can be probed - Good to do regression / Sanity testing of B as they also seem to be affected

Explanation Four
  • Assumption - Change would be affecting A, B and C Features
  • In scope - Below Cases
  • Out of Scope - A and B Features
  • This changes would require 2 days for testing
  • I would be covering below test scenarios
Let's see how this explanation can be probed - This provides the context of the QA. Level of understanding, In-scope, out of scope items. I would rate it a good estimate providing the reviewer good amount of information on estimates and coverage.

Moving a step forward in this discussion would be adding dependencies/risks

Happy Reading!!

March 31, 2011

Frequently Asked Question to QA Team

When Dev Team Takes 1 Day for Development, Why Does Test need 2 Days for Testing
  • Development involves Design, Coding, Unit Testing, Release
  • Quality of Development Deliverables can be measured by Bugs found in Test
  • Quality of Test Deliverables can be measured by Bugs found in Production
  •  With Pure black box testing, possibilities of reducing test estimates may prove risky
  • With a mix of black box and white box testing, You can take a calculated risk of Areas of Scope (Must be covered) Vs Sanity Testing in other Areas
  • Estimates should be looked at what are the areas covered in testing, Comparision of Development and Test Estimates is *NOT a* valid argument
1 Line of Code Change, Why Do you need 1 Day for Testing ?
Most common question for Test Team. What it takes to Deliver it by XX-XX-XXXX Date ?
Can we cut down Test Estimates by 1 Day ?
  • Estimates should be looked to analyze Tasks Covered, Scope of Testing, Percentage of Test Automation in place. 
  • With a mix of black box and white box testing, You can take a calculated risk of Areas of Scope (Must be covered) Vs Sanity Testing in other Areas
Why Automation did not catch this bug ?
  • Domain expertise comes with experience and exposure. Good Testers have very good domain knowledge of product
  • Obvious bugs# can be caught by automation. Layout related / Integration issues would be caught during manual exploratory testing
  • Automation plus manual exploratory testing would be good approach to maintain high Quality product
We need 100% Automation
  • Building automation framework before product lauch and Priority 1 cases Automation before product release is a good approach dealing with automation
  • Based on events captured in production, User Workflows need to converted into equivalent integration scenarios to simulate end user scenarios
  • From the production bugs# keep enhancing the Automation suite
Happy Reading!!

March 24, 2011

Selenium - Demo Test Page










Selenium 2 - Good Reads

Good Read -
Exploring Selenium 2: Introduction & Lessons Learned in Selenium 1 - Selenium RC is no more required in Selenium2, You can directly manipulate the browser.

Posting below presentation after confirming with Jason Huggins.
Selenium 2 Webinar: The Next Generation of Web and Mobile Application Testing




Hope videos are posted soon in site
Selenium 2.0 beta 1 Release
Selenium 2 in .NET framework
What is Selenium2
Web Driver Roadmap
Selenium 2.0 and web driver
Migrating to Selenium 2
It's time for QTP to Die

Happy Reading!!

March 18, 2011

TestNG Example - testng - Passing parmeters in XML File

Our next post is based on my question in stackoverflow site. Let's try an example based on Cedric reply.

Below is example code
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import com.thoughtworks.selenium.Selenium;
import static org.testng.Assert.assertTrue;
import org.testng.annotations.*;
import com.thoughtworks.selenium.*;
public class TestNGXMLData
{
@Test
@Parameters(value={"A","B","C","D"})
public void xmlDataTest(String A, String B, String C, String D)
{ 
            System.out.println("Passing Three parameter to Test "+A +" and "+B + " and "+C);
            String[] elements = D.split(",");
            for (String element : elements)
            {
                  System.out.println("Item is "+element);
                 
            }
}
}

XML for setting parameters

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite" parallel="none">
  <test name="Test" preserve-order="false">
   <parameter name="A" value="1"/> 
   <parameter name="B" value="2"/> 
   <parameter name="C" value="3"/> 
   <parameter name="D" value="4,5,6,7,8"/> 
    <classes>
      <class name="TestNGXMLData"/>
    </classes>
  </test>
</suite>

Right Click on XML and run it as TestNG Test. Below is output in console window
Passing Three parameter to Test 1 and 2 and 3
Item is 4
Item is 5
Item is 6
Item is 7
Item is 8

Extending this example and adding multiple tests and setting it for parallel execution and preserve test order.

Modified XML File
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite" parallel="classes" thread-count="5">
  <test name="firstTest" preserve-order="true">
   <parameter name="A" value="1"/> 
   <parameter name="B" value="2"/> 
   <parameter name="C" value="3"/> 
   <parameter name="D" value="4,5,6,7,8"/> 
    <classes>
      <class name="TestNGXMLData"/>
    </classes>
  </test>
  <test name="secondTest" preserve-order="true">
   <parameter name="A" value="10"/> 
   <parameter name="B" value="20"/> 
   <parameter name="C" value="30"/> 
   <parameter name="D" value="40,50,60,70,80"/> 
    <classes>
      <class name="TestNGXMLData"/>
    </classes>
  </test>
  <test name="thirdTest" preserve-order="true">
   <parameter name="A" value="100"/> 
   <parameter name="B" value="200"/> 
   <parameter name="C" value="300"/> 
   <parameter name="D" value="400,500,600,700,800"/> 
    <classes>
      <class name="TestNGXMLData"/>
    </classes>    
  </test>
</suite>

Modified code adding try-catch block (Keeping up with Coding Guidelines :))

import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import com.thoughtworks.selenium.Selenium;
import static org.testng.Assert.assertTrue;
import org.testng.annotations.*;
import com.thoughtworks.selenium.*;
public class TestNGXMLData
{
@Test
@Parameters(value={"A","B","C","D"})
public void xmlDataTest(String A, String B, String C, String D)
{ 
            try
            {
                  System.out.println("Passing Three parameter to Test "+A +" and "+B + " and "+C);
                  String[] elements = D.split(",");
                  for (String element : elements)
                  {
                        System.out.println("Item is "+element);
                       
                  }
            }
            catch(Exception e)
            {
                  System.out.println("Exception Occurred "+ e.toString());
            }
}
}

Output

Passing Three parameter to Test 1 and 2 and 3
Item is 4
Item is 5
Item is 6
Item is 7
Item is 8
Passing Three parameter to Test 10 and 20 and 30
Item is 40
Item is 50
Item is 60
Item is 70
Item is 80
Passing Three parameter to Test 100 and 200 and 300
Item is 400
Item is 500
Item is 600
Item is 700
Item is 800

Happy Reading!!

March 13, 2011

QA Skills

My Summary of QA skills for Junior/Senior QA Candidate

Junior QA
  • Follows defined QA Process
  • Prepares Bugs Reports, Status Reporting with all relevant information
  • Manages and updates automation suite
Senior QA
  • Self Organized
  • Challenges the Environment, Raise testing standards by experimenting with new processes/Adopting new Tools
  • Aware of Test Automation Framework Design and Development 
  • Sound Technical Skills 
  • Selfless - Volunteers/mentors team in terms of process/sharing best practices
  • Actively involves/drives meaningful discussions and implements best practices  
  • Takes calculated risks and delivers products meeting quality bar
Growing to a Test Architect Role
Impressive QA JD description - link


Every skill (DEV / QA / Automation) can be learned with interest and passion. This JD is a mix of DEV-QA-Automation. This is a very good write up.


One more interesting job-ad from link. Highlighted below are core skills / competencies. This co-relates to a lot of behavior traits



More Reads

March 04, 2011

Tools to evaluate for Web Testing

Time is limited, Delivering new products to customers plays a key role. Efficiency in terms of cost, time and effort is the key.

To finalize a test tool for functional/perf test automation below are few guidelines/considerations
  • Pick Top 3 test tools in terms of Features provided, Cost, Cost of Training
  • Review the tools using Trial versions/Demos
  • Analyse effort required to learn the tools, develop the framework and manage the framework
  • Evaluate cost for developing in-house tool vs Using Open Source Tools vs Licensing a Tool
Couple of Interesting tools to try out are

 More Reads

Happy Reading!!