C# - Notes - Working in XML in C# - Part X Tool Developer Notes

[Previous Post in Series - C# Basics - Tool Developer Notes - Part IX]


Tip #1 - This post is based on reading and updating XML data in C#. Example - Assume you have input as below xml file

Expected ouptut is to populate Sale value based on UnitPrice and Quantity

Steps include

• Parse the XML
• Update SaleValue based on Unit Price and Quantity
• Save the XML file

C# Example code (Console Application)

using System;
using System.Linq;
using System.Data;
using System.Xml.Linq;
namespace xmlExample
{
    class Program
    {
        static void Main(string[] args)
        {
            UpdateSaleValue();
        }
        public static string UpdateSaleValue()
        {
            try
            {
                XDocument xmlFile = XDocument.Load("E:\\SaleData.xml");
                var query = from SaleDataNode in xmlFile.Elements("Sales").Elements("SaleData")
               select SaleDataNode;
                foreach (XElement IndividualSaleData in query)
                {
                    int Quantity = 0;
                    double Price  = 0;
                    double SaleValue = 0;
                    Quantity = Convert.ToInt32(IndividualSaleData.Element("Qty").Value);
                    Price = Convert.ToDouble(IndividualSaleData.Element("UnitPrice").Value);
                    SaleValue = Quantity*Price;
                    IndividualSaleData.Element("SaleValue").Value = SaleValue.ToString();
                 }
                xmlFile.Save("E:\\SaleData.xml");
               return "0";
        }
        catch(Exception Ex)
        {
            return "-1";
        }
    }
}
}

Updated XML results

Tip #2 - For Error 'The type or namespace name ‘log4net’ could not be found'

Below link was useful to fix the issue

Happy Learning!!!

Databases Products

[Next Post in Series - Big Data Products, Big Data Updates]


Two interesting products in Database space - TeraData - Aster and Scalearc

• Using the Map reduce approach, SQL Map reduce product is available from TeraData - Aster. High Performance on large data, Real time Analytics seems impressive offering from SQL Map reduce
• Scalearc - Pattern based Caching, Load balancing, Perf monitoring available for Databases

Predictive caching 

In the context of database/reporting

• Caching the reports / avoiding DB calls
• Applying pagination to avoid database locks

In the context of Application

• Key usage time
• Key reports accessed which can be cached
• Prioritizing tasks based on users

More Reads

• IBM Debuts Netezza Analytics Appliance Geared to Retail Industry

Happy Learning!!!

C# LINQ Basics - Tool Developer Notes - Part IX

[Previous Post in Series - Tool Developer Notes Part VIII]

This post is learning's based on using LINQ to find aggregate sum from input text files. The Scenario is

• Parse tab delimited files
• Aggregate data based in few columns using LINQ queries

I am pretty much comfortable loading the data into SQL server and run TSQL queries to aggregate data in Database tables. LINQ was pretty good learning.

What is LINQ ?

Language Integrated Query. In Simple terms it provide querying capabilities in .NET languages ex-C#, TSQL queries operations - SUM, Group BY capabilites can be done in C# against a dataset

How LINQ works ?

Linq query consists of 3 parts

•  Data source (In our example it is a DataTable)
•  Query 
•  Execute Query

Command Tree is prepared and executed against data source during query execution. The Query is executed when the query variable is iterated. In below example the for loop iteration is the time when the LINQ query is executed, This is called deferred execution.

More reads Link - Link1, Link2

Coming to Exercise for the example

• Consider Input file is as per below format

Expected output contains

• Find MinDate and MaxDate based on Column2
• Aggregate SUM of Column4 based on Column2

Below Stackoverflow post provided useful directions for arriving at solution. Link1, Link2  

The steps for solution include

• Read the text files
• Load Data into a Datatable
• Run LINQ queries to aggregate data

Earlier we have tried similar approach using hash tables in previous post. Below is the example code in a C# console Application

Below is the solution code for a C# / VS2010 Console Application

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.IO;

using System.Data;

namespace LinqExample1

{

    class Program

    {

        static void Main(string[] args)

        {

            FindAggregate();

        }

        public static IEnumerable<string> ReadAsLines(string filename)

        {

            using (var reader = new StreamReader(filename))

                while (!reader.EndOfStream)

                    yield return reader.ReadLine();

        }

        public static string FindAggregate()

        {

            try

            {

                var filename = "E:\\sample.txt";

                var reader = ReadAsLines(filename);

               

                //Define Data Table

                var data = new DataTable();

                //Add Columns to Data Table

                data.Columns.Add("Column1", typeof(string));

                data.Columns.Add("Column2", typeof(string));

                data.Columns.Add("Column3", typeof(string));

                data.Columns.Add("Column4", typeof(int));

                data.Columns.Add("Column5", typeof(string));

        

                //Add Rows to Data Table

                foreach (var record in reader)

                    data.Rows.Add(record.Split('\t'));

                //Run Aggregate Query to find SUM of Column4 group by Column2

                var result_sum = from rowdata in data.AsEnumerable()

                                 group rowdata by rowdata["Column2"]

                                 into groupData

                                 select new

                                 {

                                     Group = groupData.Key,

                                     Sum = groupData.Sum((r) => decimal.Parse(r["Column4"].ToString()))

                                 };

                //List Sum Group By Column2

                foreach (var val in result_sum)

                {

                    Console.WriteLine("Column2 {0}, Total Value {1} \n", val.Group, val.Sum);

                }

                //Find Min and Max Date based on Column2

                var result_date = from rowdata in data.AsEnumerable()

                                  group rowdata by rowdata["Column2"]

                                      into groupData

                                  select new

                                  {

                                      type = groupData.Key,

                                      MinDate = groupData.Min(record => record["Column5"]),

                                      MaxDate = groupData.Max(record => record["Column5"])

                                  };

                //List MinDate, MaxDate Group By Column2

                foreach (var val in result_date)

                {

                        Console.WriteLine("Column2 Value {0}, Max Date {1} , Min Date {2} \n", val.type ,val.MaxDate, val.MinDate);

                }

                Console.ReadKey();

                return "0";

            }

            catch (Exception EX)

            {

                return "-1";

            }

        }

    }

}

Below is the actual output

More Reads

Query Expression Syntax Examples: Aggregate Operators (LINQ to DataSet)

Happy Learning!!!

Tool Developer Notes - Part VIII

[Previous Post in Series - Tool Developer Notes Part VII]

Next Post is related to parsing tab delimited flat file and aggregate data from selected columns
Input format listed below

Expected output is

Steps for solving the problem

• Parse a text file, aggregate selected column data and show aggregated results
• Implemented using hashtables in C#
• Update the values if key already exists for them

C# Example code. Please add necessary namespace to run this program, reusable method and example code listed below

using System;

using System.IO;
using System.Data;
using System.Collections.Generic;
using System.Collections;

Code Snippet

public static string parsefiledata()

{

    char[] delimiterChars = { ' ', ',', '.', ':', '\t' };

    Hashtable AggregatedValues = new Hashtable();

    try

    {

        //Open Text File

        using (StreamReader sr = File.OpenText("E:\\sample.txt"))

        {

            string line;

            TextWriter tw = null;

            int newval = 0;

            while ((line = sr.ReadLine()) != null)

            {

                //Split it based on tab delimited spaces

                string[] Attributes = line.Split(delimiterChars);

                //Verify if Key Already Exists

                if (AggregatedValues.ContainsKey(Attributes[1]))

                {

                    newval = Convert.ToInt32(AggregatedValues[Attributes[1]]);

                    newval = Convert.ToInt32(Attributes[3]) + newval;

                    AggregatedValues[Attributes[1]] = newval.ToString();

                }

                //Add new Key if it does not exist

                else

                {

                    AggregatedValues.Add(Attributes[1], Convert.ToInt32(Attributes[3]));

                }

            }

        }

       

        foreach (string key in AggregatedValues.Keys)

        {

            Console.WriteLine(String.Format("{0}: {1}", key, AggregatedValues[key]));

        }

        Console.ReadKey();

        return "0";

    }

    catch (Exception ex)

    {

        return "-1";

    }

}

Console window output...

Happy learning!!!

Tool Developer Notes - Part VII

[Previous post in series - Tool Developer Notes - Part VI]

Series of posts on learning's from developing tools using .NET platform. This post is about XML validation. Validating xml is based on defined format. Detecting in case there is a missing node / format issues. Define a schema for the xml format and validate it against the xml data

Step 1 - Define xml data format. Sample format provided below

Step 2 - Generate schema from the xml

Step 3 - .NET code snippet to validate the input xml format. stackoverflow approach is reused, Read schema from XSD filepath and input filepath is modified for this example. Sample Console Application

using System;
using System.Windows.Forms;
using System.Collections;
using System.Collections.Generic;
using System.Configuration;
using System.Xml;
using System.Xml.Schema;
using System.IO;
namespace TestExample
{
    class Program
    {
        public static void Main(string[] args)
        {
            ValidateXSD();
        }
        public static void ValidateXSD()
        {
            try
            {
                XmlReaderSettings settings = new XmlReaderSettings();
                settings.Schemas.Add(null, @"C:\Users\Desktop\XSD Post\Input.xsd");
                settings.ValidationType = ValidationType.Schema;
                settings.ValidationEventHandler += new System.Xml.Schema.ValidationEventHandler(ValidationCallBack);
                var reader = XmlReader.Create(@"C:\Users\Desktop\XSD Post\Input.xml", settings);
                // Parse the file. 
                while (reader.Read());
            }
            catch (Exception EX)
            {
                MessageBox.Show(EX.Message);
            }
        }

       // Display any warnings or errors.

        private static void ValidationCallBack(object sender, ValidationEventArgs args)
        {
            if (args.Severity == XmlSeverityType.Warning)
                MessageBox.Show("\tWarning: Matching schema not found.  No validation occurred." + args.Message);
            else
                MessageBox.Show("\tValidation error: " + args.Message);
        }
    }
}

Step 4 - In case of errors the actuall error would be caught in exception block

More Reads

Quick Aside on XML - The Wrong Way to Use XML


Happy Learning!!!

Tool Developer Notes - Part VI

[Previous Post in Series - Tool Developer Notes - Part V]

This post is based on learning on Adding SOAP Header username and password using WSE 3.0. This stackoverflow link was useful with clear steps

There were some challenges implementing the steps on VS 2010 and Win7 Machine

• Download and Installed WSE 3.0 from link
• Generating Proxy Class Command is provided in link

• Regenerate proxy class with wsewsdl3.exe with the /type:webClient

• While Trying to Generate Proxy using WseWsdl3.exe encountered error 
• wsewsdl3.exe System.InvalidOperationException: Could not get the install directory for .NET Framework v2.0 SDK. Please install the .NET Framework v2.0 SDK.
•  Fix for this was mentioned in link
•  Add sdkInstallRootV2.0 in registry in below path

 

•  Proxy class is generated successfully after this change

• Difference between proxy class generated between WseWsdl3.exe and wsdl.exe is
• Wsdl.exe generated proxy class for web service implements - System.Web.Services.Protocols.SoapHttpClientProtocol
• WseWsdl3.exe generated proxy class for web service implements - Microsoft.Web.Services3.WebServicesClientProtocol

More Reads

How to: Access a WSE 3.0 Service with a WCF Client
How to Host WCF Service in BasicHttpBinding & HTTPS
One more learnings on the same - ASMX services and Self-Signed Certificates

Happy Learning!!!