Database Reads

Good Read - 6 Part Series on CAP Theorem

An analysis of the CAP Theorem

CAP Theorem Part 2: What is the CAP Theorem?

CAP Theorem Part 3: Definitions and terminology

CAP Theorem Part 4: The relationship between Consistency, Availability, and Partition Tolerance

CAP Theorem Part 5: Implications of the relationship between Consistency, Availability, and Partition Tolerance

Cap Theorem Part 6: Conclusion


Happy Reading!!!

Database Internals [Good Reads]

Please download and read white paper 'OLTP Through the Looking Glass, and What We Found There'

The whitepaper provides more details on current architecture of OLTP RDBMS Systems. I can closely relate it with MSSQL Server. It covers the key architecture details. The Paper evaluates options by eliminating some of the features based on current hardware systems and demonstrates significant performance gains

 

At a beginner level some of architecture keywords / concepts good to learn are

• WAL (Write Ahead Logging)
• Log Manager
• Buffer Manager
• Two Phase Locking
• Concurrency 
• Eventual Consistency
• Transaction Manager

I am posting some relavant simple explanations of the same based on MSDN documentation. WAL -

 

Write Ahead Logging (Source - Link)

 

 

Buffer Manager - Page is fetched to main memory through Buffer Manager (Source - Link)

  

Two Phase Locking - A detailed note of how transaction is executed from Brent Ozar Post

 

Concurrency Control - Governed by Isolation levels, Earlier discussed in same blog

 

Log Manager - Manages Logging, WAL

 

Eventual Consistency - Over a period of time the data will be consitent across different nodes (databases) distributed over network. Read More @ Consistency models in nonrelational dbs 

 

 

More Reads  

• ColumnStores vs. RowStores: How Different Are They Really?
• Materialization Strategies in a Column-Oriented DBMS
• Query Execution in Column-Oriented Database Systems

 

Happy Reading!!! 

HBASE & Berkeley DB Internals [Good Reads]

• Berkeley DB - Architecture and Implementation Details
• ACID properties of HBase
• Apache HBase Write Path
• Apache HBase Internals: Locking and Multiversion Concurrency Control
• Columnar Storage Query Execution Basics

Happy Reading!!

Good Tech Talk - Making Better Software - QA & DEV

Today I checked this tech talk. This is very very good. **MUST VIEW** for passionate DEV / QA / IT Pro. Complete Tech Talk is inspiring, offers a lot of improvement ideas

 

Make Better Software: The Training Series - Module 2

Between 24:31 to 30:52 minutes

Very Good Joel talk on test skills, testing competency and also covers **Agile QA**

 

Summary of QA Talk

• Benefits of Having QA in a Team
• Benefits of Rapid Feedback between Dev & QA
• Tools for testing + Functional testing both are critical

Between 32:34 to 34:37 minutes

• How Writing code vs Developing a Great product differs?

Between 44:18 to 48:54

• Intrinsic and Extrinsic motivation
• Intrinsic Motivation matters!!

 

Hope you enjoyed the tech talk. Happy Listening!!!

Good GTAC Testing Talks

GTAC 2011: The Latest in Google Test Tools

Tools covered in the session are BITE and Quality Bots

BITE - Browser Integrated Test Environment

• Record Browser Actions, File Bugs Automatically without switching from your browser, Replay Recorded Actions
• BITE is available as a Chrome Extension 
• Records User Actions @ Browser Level
• Track Actions while finding UI bugs during Exploratory Testing  
• Bug Filing from Browser itself
• Include Screenshots, Repro Steps while filing bug

Quality Bots - Automated Layout Issues Detection in Chrome browsers

With the Tools Implemented QA Cycle works as below

• Run Quality bots to detect layout issues
• Allow exploratory testing for Manual testers and capture test steps, screen shots, related data using BITE chrome extensions for bugs identified
• Automated Bug Filing using BITE from browser itself

Tools + Exploratory Testing with test steps captured makes it easier to reproduce bugs

More Interesting Reads

• GTAC 2011: Automating Hadoop Stack Deployment and Testing
• GTAC 2011: WebDriver
• BITE

Happy Learning!!!

  

2012 Big Data Interesting Milestones

2012 was a fantastic year in terms of Big Data and Cloud Adoption. Massive Big Data focus and Products focussed on Big Data Analytics. Fortunately the world did not end as per Mayan Prediction.

 

How Traditional Players catch up in Big Data Space

• Traditional RDBMS Vendors MSSQL, ORACLE offering their own NOSQL Solutions, Hadoop Connectors to leverage the power of Hadoop and enhance their BI offerings
• IBM BigInsights Hadoop based Analytics Platform, Informatica and Microstrategy enhanced their Big Data Capabilities this year by providing integration with Hadoop Ecosystem
• Acquire Hadoop based Startups to catch up with Big Analytics

Real Time Analytics Trend in 2012 

• Cloudera launched Impala platform to provide Real Time Analytics
• Real Time Analytics based on Twitter Storm

Big Data Fifth Elephant Conference provided good insight on Big Data Use Cases and Adoption rates

Big Data Products

• Good Read - Big Data - Promising Startups
• Amazon Announced Redshift DataWarehousing as a service
• Google Launched BigQuery, Cloud Business Intelligence Service for Big Data

Indian Startups Working on Big Data / Leveraging Hadoop Ecosystem for Analytics

• Inmobi
• Qubole
• Flipkart

2013 Focus 

• Big Data Analytics Adoption across Industries (Verticals)
• Increased Implementation / Adoption of Big Data Products
• Columnar databases gaining more focus (Vertica, H-Store)
• Databases to watch out in 2013 - Hadapt (HadoopDB), H-Store(OLTP), F1(Google Distributed RDBMS) 

Interesting Big Data Reads

• Classifying Today’s “Big Data Innovators”
• The future is now: 10 startups leading the way in ‘big data’
• Major Data Warehousing Events of 2012 (and Predictions for 2013)

Inspiring Tweet to end the post



Happy Learning!!!

  

Python Basics

Two Coursera classes (An Introduction to Interactive Programming in Python, Learn to Program: The Fundamentals) provide very good python beginner learning materials. Python is a Interpreted Language

Quick Summary and Python Notes

• Guido van Rossum is author of python, Good Wikipedia read about him in link  
• Python Practice Book
• Notes from Advaced Python Workshop
• Online Editor available at CodeSkulptor
• Also check Very Good Question Is Python Interpreted or Compiled?
• Good Book to Get Started - A Byte of Python -Swarooph
• Learning Python the hard way

Basic Examples and beginner notes posted based on course notes. All Code can be tried in CodeSkulptor.  There are a few syntax differences but this is similar to C programming language.

Example1 - Arithmetic Expressions

• Basic Arithmetic Expressions
• Variables, Assignments, Sample Exercise in link

All Scripts can be run in FireFox / Chrome in CodeSkulptor (Not in IE)

Example 2 - Working with Functions
Built In functions - max, abs

Example 3 - Custom Functions

• Example function to sum up two numbers

Example 4 - If Statement - Conditons , Else If and Else Operator

Example 5 - For Loop - Iteration

Example 6 - While Loop iteration

Happy Learning!!!

Tool Developer Notes

Tip #1 - Sending Email on Error using Log4NET

Tip#2 - Append XML Node in C#

Before Code Execution

Post Code Execution

using System;
using System.Globalization;
using System.Data;
using System.IO;
using System.Xml;
namespace ExampleCode
{
    public class ExampleCode
    {
        static void Main()
        {

            //Open XML
            XmlDocument xmlPreCountFileData = new XmlDocument();
            xmlPreCountFileData.Load("E:\\abc.xml");
            XmlElement elmXML = xmlPreCountFileData.CreateElement("Node");

            //Append Node
            //Append Existing XML Node
            string xmlDataElement = @"<A>40</A>";

            elmXML.InnerXml = xmlDataElement;
            xmlPreCountFileData.DocumentElement.AppendChild(elmXML);
            xmlPreCountFileData.Save("E:\\abc.xml");
            Console.ReadLine();
        }
    }
}



Tip #3 - Thread Enhancements in .NET 4.0

Threading made easy in .NET 4.0
C# Mulththreading Improvements in .NET 4.0
.NET 4.0 and System.Threading.Tasks


Happy Learning!!!

Big Data Trends

Excellent Presentation on Big Data Trends..

Big Data Trends from bigdatalandscape

Happy Reading!!!

Hadoop Learning questions from Quora

Quora has a good collection of questions on Hadoop ecosystem, architecture and beginner level questions. Bookmarking some of interesting questions

• HBase: Why does HBase depend on HDFS?
• How much time did it take to develop Hive at Facebook?
• Good Notes from Workshop on Algorithms for Modern Massive Data Sets
• What is the best way to learn Hadoop and Cassandra for a starter?

Recent Updates

• Cloudera Impala: Real-Time Queries in Apache Hadoop, For Real
• Does Cloudera Impala have any drawbacks when compared with Hive?
• How does Cloudera Impala compare to Vertica?

More Reads

• HBase setup (0.20.0)
• Below is a code sample of how to read from and write to HDFS in java
• Get a hadoop cluster running in 20 Minutes
• Hadoop and Hive
• How to Load Text file into Hbase in Linux Machine

Happy Learning!!!