"No one is harder on a talented person than the person themselves" - Linda Wilkinson ; "Trust your guts and don't follow the herd" ; "Validate direction not destination" ;

March 24, 2012

Tool Developer Notes - Part V

[Previous Post in Series - Tool Developer Notes - Part IV]

This post is based on work done in last couple of weeks

Tip #1 - While trying to run a Winforms Exe, I used to get below message prompt in my Win7 Machine

After a little google effort MSDN forum question was useful to fix this warning. Workaround is edit the manifest file in Debug folder which contains the Exe file

Added below lines of code

This fixed the message.

Tip #2 - C# File Exists Check

Tip #3 - C# Directory Exists Check

Tip #4 - C# - Check Return Value in SQL Command Return

In the earlier post on Tool Developer Notes Series I was not checking if the return value is NULL,
  • Modifying the example to check returned object is not null
  • Simple Windows Console Application
  • Add the references as per mentioned example code snippet
using System;
using System.Collections;
using System.Collections.Generic;
using System.Configuration;
using System.Data;
using System.Data.SqlClient;
using System.IO;
using System.Linq;
namespace TestExample
{
    class Program
    {
        public static void Main(string[] args)
        {

            Program P = new Program();
            P.FetchScalarValue();
        }

        public void FetchScalarValue()
        {
            try
            {
                Console.WriteLine("Fetch Single DB Values Example \n");
                SqlCommand comm = new SqlCommand();
                comm = SetUpConnection();
                OpenConnection(comm);
                comm.CommandType = CommandType.Text;
                comm.CommandText = @"SELECT Top 1 FirstName
                                  FROM [AdventureWorksLT2008R2].[SalesLT].[Customer]";
                object result;
                result = comm.ExecuteScalar();
                if(result!= null)
                {
                    Console.WriteLine("Query Result is " + result.ToString());
                }
                Console.ReadLine();
                Console.ReadLine();
            }
            catch (Exception EX)
            {

            }
        }
        public SqlCommand SetUpConnection()
        {
            string strConn = ConfigurationManager.AppSettings["ConnectionString"];
            SqlCommand comm = new SqlCommand();
            comm.Connection = new SqlConnection(
                strConn);
           return comm;
        }
        public void OpenConnection(SqlCommand comm)
        {
            comm.Connection.Open();
        }
        public void CloseConnection(SqlCommand comm)
        {
            comm.Connection.Close();
        }
    }
} 

App.config file Entry


<?xml version="1.0" encoding="utf-8" ?>
<configuration>
      <appSettings file="" >
            <clear />
            <add key="ConnectionString" value="Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=AdventureWorksLT2008R2;Data Source=.\SQLSERVER2008R2" />
      </appSettings>
</configuration>


Happy Learning!!!

March 17, 2012

TSQL Enhancements in SQL 2012 - Part III

[Previous Post in Series - TSQL Enhancements in SQL 2012 - Part II]

This dedicated post is for SQL 2012 Paging feature. This is a very important feature in terms of performance impact. I have seen pagination playing a crucial load while data loading. This inbuilt paging support would help to fetch only required data.

It would be intersting to see how paging is implemented by the Database Engine. How execution Plan looks like when you run a paging query.
Scenario
  • Create a Table
  • Populate a million records
  • Add Indexes on columns
  • Run Paging queries
  • Check Execution Plan
Step 1 - Creating Table and Populating Data
Step 2 - SELECT query with Filter Condition using Primary Key Column. As you can see it is a Clustered Index Seek Operation
Step  3 - Implement the Same using Paging Option. Check the Execution Plan  



Step 4 - Results of Paging Query

Since We are requesting pagination on a clustered index column, I expected this must be a seek but it seems to be a SCAN.This might be a costly operation. I hope to post answer for it in next set of posts.


More Reads
Happy Learning!!!!

TSQL Enhancements in SQL 2012 - Part II

[Previous Post in Series - TSQL Enhancements in SQL 2012 - Part I]


SEQUENCE

  • This feature is similar to Identity Concept
  • Normally an Identity Column is set automatically, In case of Sequence the value is set in TSQL code level
  • Similar to Identity You can set increment value, set max value
  • From the Virtual Lab training material it is mentioned - Recommened scenario is when you need the value before loading in Table, Increment, Set Max Value limit
Scenario
  • Create a Sequence from Start Value - 0, Increment by 5, Max Value - 50, Again Reset Start from 0

    Format 

    • Specify Format. Reusing Date Examples from Post

    TRY CONVERT

    • TRY to Convert Value, On Success Value returned else NULL
    • USEFUL when you do row by row operation and if one of value exceed limit (ex- assign int variable big int value / string value)
    Paging - Pagination of Results, Very Nice Feature, We will try this in next post..

    Happy Learning!!!!!

    March 16, 2012

    TSQL Enhancements in SQL 2012 - Part I

    [Next Post in Series - TSQL Enhancements in SQL 2012 - Part II]

    I had taken Session on SQL 2008 TSQL Enhancements, SQL 2008 R2 TSQL Enhancements in my earlier roles. This post is to cover TSQL Enhancements in SQL 2012. Online SQL Server Virtual Labs is available in Link. You should give it a try. I have tried couple of exercises for Enhancements in TSQL for SQL Server 2012
    • Throw Feature Support in Try-Catch
    • IIF - Simplify IF-ELSE Logic
    • CHOOSE - Select a value from a list based on index of value
    • CONCAT - String Concatenate Feature
    Throw Keyword Support

    Feature addition is Throw keyword is supported to throw exception. This is useful for debugging, You can set throw option to debug. For logging you can use catch option and log errors for production database

    Normal Try-Catch Scenario


    With Throw Keyword
     
    Concat
    • Concatenate Column Values, Useful for Results Reporting Purpose


    IIF and CHOOSE
    • IIF simplifies IF else condition specifying both in a single line
    Example - Maximum of 3 numbers

    CHOOSE
    • Return based on index value specified
    Next Post we would cover
    • TRY_CONVERT
    • Format
    Happy Learning!!!!

    Tool Developer Notes - Part IV

    [Previous Post in Series - Tool developer notes part III]
    Tip #1 - Initially I used to read IP address from App.config File for my C# Application. Based on feedback I moved the URL assigment in implementation. Only IP Address and Port would be read from App.config

    While trying to compile below is error. Error - C# + System.UriFormatException: Invalid URI: A port was expected because of there is a colon (':') present but the port could not be parsed.
    Below link was useful for the answer. Let's try out that answer

    Step 1 - A simple Windows Forms App

    Under App.Config App setting add following keys

    <!--IP Address-->
    <add key="IPAddress" value="117.00.162.110"/>
    <!--Port-->
    <add key="Port" value="1433"/>

    Step 2 - Add Windows Configuration Reference and System.Web Reference



    Step 3 - For the button click add below code in the method

    string IpAddress = ConfigurationManager.AppSettings["IpAddress"];
    string Port = ConfigurationManager.AppSettings["Port"];
    string TestUrl = string.Format(@"http://{0}:{1}/test/services/testnode/", HttpUtility.UrlEncode(IpAddress),
    HttpUtility.UrlEncode(Port));
    Uri U = new Uri(TestUrl, UriKind.Absolute);
    MessageBox.Show(U.ToString());

    Using HttpUtility and Uri for setting up URL fixed the issue. If the URL is relative you need to use relative option while setting up new uri (uniform resource identifier). Also note there is a double '//' after the http. This is also very important.

    Tip #2 - Log4Net logging had duplicate entries created in logs. This was due to configuration setting in app.config file. Stackoverflow answer was useful to correct duplicate logging issue. Lets try a sample code for Log4Net logging. Earlier example please check in link

    Modified App.Config File

    <?xml version="1.0" encoding="utf-8" ?>
    <configuration>
    <configSections>
    <sectionname="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler,log4net" />
    </configSections>
    <appSettingsfile="" >
    <clear/>
    <addkey="log4net.Internal.Debug" value="false"/>
    </appSettings>
    <!-- This section contains the log4net configuration settings -->
    <log4netdebug="true">
    <appendername="LogFileAppender" type="log4net.Appender.FileAppender">
    <layout type="log4net.Layout.XMLLayout" /> -->
    <param name="File" value="Log4NetExample.log"/>
    <param name="AppendToFile" value="false" />
    <layout type="log4net.Layout.PatternLayout">
    <header type="log4net.Util.PatternString" value="[START LOG] %newline" />
    <footer type="log4net.Util.PatternString" value="[END LOG] %newline" />
    <conversionPattern value="%d [%t] %-5p - %m%n"/>
    </layout>
    </appender>
    <!-- Setup the root category, add the appenders and set the default level -->
    <root>
    <level value="INFO" />
    <appender-ref ref="Console" />
    <appender-ref ref="LogFile" />
    </root>
    <!-- Specify the level for some specific categories -->
    <loggername="log4NetExample" additivity="false">
    <!-- <appender-ref ref="B" /> -->
    <level value="INFO" />
    <appender-ref ref="LogFileAppender" />
    </logger>
    </log4net>
    </configuration>

    This config changes to previous example fixed the duplicate logging issue. I still need to explore all the Log4Net settings. Hoping to try them out in next set of posts.
      
    Tip #3 - This is for error - "The AXIS engine could not find a target service to invoke!  targetService is ABC/". When I was trying to invoke a service through proxy code i received this error. The reason is there was a front slash '/' which need to be removed to fix this issue. There were so many answers provided for this error. Hint to remove slash was useful from answer

    Tip #4 - Read Text from a File and Encode it in UTF8 format

    Tip #5 - Reading a File till end

    Tip #6 - Why do I get the error "Object reference not set to an instance of an object"?


    Happy Learning!!!

    March 04, 2012

    Big Data - Basics - Getting Started

    [You may also like - NOSQL - can it replace RDBMS Databases?]

    This post is towards learning fundamentals and evolution of big data computing. Based on my discussion with one of my colleague. The quest to find the details of big data. Where is started, why it is needed, What is the current state of Big Data Computing ?


    Why Big Data ?
    • Distributed Data processing, Supporting Massive Data (Peta bytes), Scalability were challenges in traditional BI systems (MSSQL, ORACLE and other BI solution providers)
    • An alternative to Transaction processing systems based on ACID properties, fixed Schema design, scalability issues led to evolution of NOSQL Databases, Hadoop based systems
    • Search engines, social networking sites accumulate large amounts of data in a very short time
    • Scalability, flexible schema support, indexing support are properties of NOSQL systems
    • Moving away from traditional ETL based data processing which took alot of time to consolidate various data sources and process large amount of data
    Phases Involved in Traditional BI Processing
    MSBI
    • ETL Processing
    • Build Data Marts
    • Build Cubes
    • Run SSRS reports 
    Phases Involved in Big Data Processing
    • Storage can be Hadoop based / NOSQL Based. It would be useful to check on evolution of Hadoop.
    • Detailed BI processing in Hadoop. Presentation Realtime BI in Hadoop is useful
    • Some important metrics in data processing in big data. Yahoo processed 1 TB data in 16 Secs, 1 PB data in 16 Hours (Source - Link - Slide 29)
    • Hadoop Vs RDBMS (Slide - 17 of presentation is good) 
    How Big Data Evolved
    • Everything started with google map/reduce approach. Followed by Hadoop evolution by 2006
    • Yahoo, Facebook, twitter and other major players opted for Hadoop based databases, NOSQL databases
    • Reference - link was useful
    How Map reduce works
    • Input data is converted (reduced) into meaningful key / Value pairs
    • Since data from source is in processed (reduced) there is no need to load data and process it at the server level
    • This reduced data is consolidated from various sources is used for Data Analytics / further Data processing (Data Marts etc..)
    • Post is very good note in simple terms to learn Map / Reduce implementation - Map reduce includes (Distributed Data Processing, Data stored as Keys)
    • The program mentioned word count is available in link
    How Hadoop works
    • Hadoop is a Framework for Distributed Data Processing
    • Based on Map Reduce Approach
    • Slide 9 of presentation is very good representation of Hadoop setup.
    • The Key components include HBASE for storage, HIVE - Query language for Hadoop
    • SQOOP - Import data from RDBMS systems to Hadoop Clusters, Pig, Avro etc..
    • Good Presentation - link
    Summarizing Key points on Hadoop Usage
    • Suitable for Data Mining, Analytics from Unstructured data
    • Not Recommened for RDBMS compliant systems - Banking, OLTP based systems, financial systems etc..
    How Microsoft & Oracle play with Big Data
    Startups in Big Data space

    • NUODB - Cloud based RDBMS compliant database. Capable of large data processing and a competitive player in big data space
    • SPIRE - Based on Hadoop and HBASE. Real time scalable database
    • Rethink DB - Key Value pair based Storage
    • Emergence of Columnar Database, Vertica Ranked No.1 for Columnar Data
    More Reads 
    Planning to get started with Hadoop, NuoDB in coming weeks....

    Another Excellent Articles Collection List from Wikibon
    Big Data: Hadoop, Business Analytics and Beyond
    Real-Time Data Management and Analytics Come in Many Flavors
    Big Data Market Size and Vendor Revenues
    Microsoft is BIG in Big Data

    Happy Learning!!!

    February 24, 2012

    Note on success of GUI automation

    There are plenty of blogs which recommend
    • When to Automate, At what stage of project execution (working feature set, stable application etc..)
    • What to Automate, How to identify a test case for automation (repeatable steps, stable feature, regression cases, BVT cases etc..)
    This post is a example on determining quality of automation suite. This stackoverflow answer is my motivation for this post
     
    Joe Strazzere answer is simple, straight forward and portrays real world scenario. Below is snap of the answer underlined important lines in my perspective
     
     
    What to focus on Automation 
    • Tests focusses on basic scenarios and slowly extending to cover end to end functionality
    • Execute 10~15 core functional scenarios
    What not to focus on automation
    • Merely focussing one existence of ids
    • Testing the front end without connecting underlying layers
    What is good automated test case ?
    • Login to online shopping portal
    • Search for products (query the DB to fetch products)
    • Invoke web service involved to fetch results
    • Compared returned results in UI vs Web Service Results
    What is not efficient automated test case ?
    • Login to online shopping portal
    • Search for products which are hardcoded
    • Compared returned results in UI with predetermined hardcoded results
    If you find your automation tests pass and your manual test identifies bugs then it is a indicator for automation test cases improvement. What matters is Quality not Quantity !!
     
    Happy Learning!!! 

    February 11, 2012

    Telerik Test Studio Webinar

    There are lot of test Tools available in market. I attended Telerik Test Studio session. The tool is focussed on Microsoft Platform (Web Testing WPF/Silverlight). Session was useful to understand Product Portfolio, features

    Notes from the Session
    • Overview and benefits of test automation covering rerunning test cases, reduce test execution efforts, eliminate human error etc..
    Test Studio
    • Inbuilt Scheduling Engine
    • Performance Test Support
    • Integration withing TFS
    • Cross browser testing support
    • Microsoft Platform focussed test product (WPF/Silverlight)
    • Data Driven Testing Support
    More Details - Link
     
    Express Edition Features
    • Available as a VS plugin
    • Generate unit test cases which can be run using NUnit, XUnit, MbUnit
    Demo
    • Record and playback of Web Test was demonstrated in the session
    • This made me to remember VSTT web test record and play back feature
    It would be interesting to download and Evaluate Telerik Free Testing Framework
     
    Happy Learning!!

    February 09, 2012

    Tool Developer Notes - Part III

    [Previous Post in Series - Tool Developer Notes - Part II]
    [Next Post in Series - Tool Developer Notes - Part IV]

    Adding few more interesting notes based on learning's

    Tip #1 - Working with DateTime Picker in C#. Requirement was to write in specific format. Provided below is an example using WinForms Application


    For the Submit button try out the sample code for Setting date, Adding Duration

            private void btnSubmit_Click(object sender, EventArgs e)
            {
                int duration = 0;
                DateTime CurrentTime;
                String CurDate;
                duration = Convert.ToInt32(txtDuration.Text);
                CurrentTime = dTPTimeSet.Value;
                for (int i = 0; i < 10; i++)
                {
                    CurDate = CurrentTime.ToString("MM/dd/yyyy hh:mm");
                    MessageBox.Show(CurDate);
                    CurrentTime = CurrentTime.AddMinutes(duration);
                }
            }


    Tip #2 - Check if string is GUID. Tweaking the code from Stackoverflow question

    public static bool IsGuid(string guid)
            {
                if (!string.IsNullOrEmpty(guid))
                {
                    Regex regex = new Regex(@"^(\{{0,1}([0-9a-fA-F]){8}-([0-9a-fA-F]){4}-([0-9a-fA-F]){4}-([0-9a-fA-F]){4}-([0-9a-fA-F]){12}\}{0,1})$");
                    return regex.IsMatch(guid);
                }
                return false;
            }

    Tip #3 - Loop through all files in a directory
    How to loop through all files in a folder using C#

    Tip #4 - Last File Created time in a directory
    using System.IO.DirectoryInfo we find last file created time in a directory

    string Folder = @”C:\Test”;
    DirectoryInfo dinfo = new DirectoryInfo(Folder).GetDirectories("*", SearchOption.AllDirectories).OrderByDescending(d => d.LastWriteTimeUtc).First();
    //Find the Last File Created Time in Directory
    DateTime LastCreatedTime = dinfo.LastAccessTime;

    Happy Learning!!!