Agile!! Agile!! Agile!! Will it be Fragile!!

Pair Programming- Two People working together on same problem at same time/place. Advantages-better code, code review, Share knowledge, Less Probaility of Injecting bugs. Pairing Types (Beginner & Beginner, Beginner & Expert, Expert & Expert). Intro video. Can it be Three people working together PM-DEV-Test all three of them sit together and code/Test it. Probably it might work for small applications.

Startup mode -  Work without everything having to be spelt out.


XP - Focus on Small Releases/Iterations. Deliver business value in every iteration. XP Intro

TDD - Developer writes automated test cases for the desired function. Then he codes for actual functionality. TDD Intro. Can it be Developer writes code, Tester writes automated code for the test. When dev is ready with code, Test is ready with automated code. Test & Dev Phase are integrated. Test-Driven C#

Kanban Software Development - A modified SCRUM methodology.

Kanban comes from the Japanese for “visual card”.

Comparision is provided
Scrum batches work in fixed iterations - Kanban is typically (but not always) iterationless
Scrum has no work-in-progress limits - Kanban always limits WIP
Scrum can have large backlogs - Kanban discourages large queues.
Scrum measures velocity after each iteration - Kanban uses limits to adjust productivity in real-time
Source - Link

Whatever process or model that comes into market. A clear BRD and a Clear understanding of intended functionality by DEV-TEST-PM only can provide a good code that meets business demands. With the expectations set by business, we can target for minimal working software and iterations on top of it. Writing a Hello World program can follow all of the above methodology but not for an enterprise application. Not all Process would suit every project. I am not discouraging these processes but a reality check need to be done on how effective they can help a project.

1. Document and communicate clearly what can be delivered in the minimum period (Weeks/Days) - Release Cycles

2. Business needs working software that helps to run a business. Incomplete functionality delivered in days is worse than complete functionality delivered in weeks (Product Size & Complexity)
3. Quality Speaks first than Time taken for delivery. Know what you code for. (Agility Vs Stability)
4. Don’t let your estimates become commitments. Remember the difference between an estimate and a commitment and keep the two activities separate. Link
5. Prioritize-Gather-Manage Changes during Release Cycles

I agree with the definition of Agile posted in link

• Deliver a continuous stream of potentially shippable product increments
• At a sustainable pace
• While adapting to the changing needs and priorities of their organization

A couple of useful reads
How I Learned to Program Manage an Agile Team after 6 years of Waterfall
Iteration Planning Meeting with CodePlex
How Agile Works – My Program Manager Cheat Sheet
The philosophy of Kanban is Kryptonite to Scrum
Interesting read - Software's Classic Mistakes--2008

With Agile SCRUM model below items, you can also observe

• Confusing estimates with targets and stretching to meet dates
• Excessive multi-tasking
• Research-oriented Development (Underestimation of unfamiliar tasks)

2009 State of Agility Results  - Lists Reasons for following Agile
Excellent Suggestion on Test Role in Agile Projects
7 Practices to Agile QA

• Test Involves in all Phases - Reqmts, Design, Code, Testing
• Prioritize Test Cases
• Target and Improve Automation Testing
• Encourage Peer-Reviews (Design/Code/Test Scenarios)
• Encourage Pair Programming Developer + Test (Act as SME)

Agile isn’t always Agile - Excerpt - "Just turns into a micro-management environment, where devs have to defend their daily work. Of all the work environments I hate the most, micro-management environments are THE worst. I don’t like working in them, and I don’t like creating them"

Lisa Crispin talks about the 'Trends in Agile Testing'
Agile Development - Evolutionary Design
'Releasing to Production Every Week'
My 7 principles to design the architecture for a software project.
A Simple, Definitive Metric of Software Development Productivity

Nice pic from http://www.utest.com/ presentation...

Great Read from Quora Read 
Quote of Jeff Nelson's answer to Why is it that when "pair programming" produces better code, almost no company practices it? on Quora

Happy Reading!!!

Code Readbility and Performance

Its always good when TSQL code we write with proper indentations, naming conventions, readability. Equally to all this points the code should be performance compliant. I found this post good one. I want to try the same. We have a table with 3 columns. All columns indexed. We need to make a search based on the three columns.

Code Required to setup our demo
USE TEMPDB
CREATE TABLE TestforCode
(
     ID INT IDENTITY(1,1) PRIMARY KEY,
     NAME VARCHAR(100),
     CITY VARCHAR(100)
)

DECLARE @I INT
SET @I = 1
WHILE 1=1
BEGIN
    INSERT INTO TestforCode(NAME, CITY)
    VALUES (('NAME ' + CONVERT(CHAR(20),@I)),('CITY ' + CONVERT(CHAR(20),@I)))
    IF @I > 10000
    BREAK;
    SET @I = @I+1
END

CREATE INDEX IX_TestforCode_City ON TestforCode(City)
CREATE INDEX IX_TestforCode_Name ON TestforCode(Name)

Input is all 3 columns I need to search based on it, Below two Queries would serve the purpose.
Query#1
DECLARE @ID INT, @Name VARCHAR(100), @City VARCHAR(100)

SET @ID = NULL
SET @City = 'City 1'
SET @Name = NULL
IF @ID IS NOT NULL
    SELECT * FROM TestforCode WHERE (ID = @ID)
ELSE
    IF @Name IS NOT NULL
    SELECT * FROM TestforCode WHERE (NAME = @Name)
ELSE
    IF @City IS NOT NULL
    SELECT * FROM TestforCode WHERE (CITY = @City)

Query#2
DECLARE @ID INT, @Name VARCHAR(100), @City VARCHAR(100)
SET @ID = NULL
SET @City = 'City 1'
SET @Name = NULL
SELECT * FROM TestforCode WHERE (ID = @ID) OR
(NAME = @Name) OR (CITY = @City)

Looking at this Query#2 looks sleek and simple but what about performance. End of Day You query should be optimal in terms of CPU, IO usage. Lets see Execution Plan for both Queries. Ctrl Key+M is the command to get actual execution plan.

Query#1











Query #2







Query #1 is the optimal query which uses non-clustered index on city column and a bookmarklookup to fetch other records. As much as code readability it also need to be performance compliant code.


More Reads

Top 10 Developer Mistakes That Won't Scale
Transact-SQL Tips and Tricks


Happy Learning!!!

SQL Tip

I wanted to simulate range locks. I tried some examples in web, I couldn't actually repro it. Roji suggested below repro steps. We can see range locks and also why you see range locks in this scenario.

Created a Test Table and Populated Test Data as below
USE TestDatabase
IF OBJECT_ID ('TestLocks') IS NOT NULL DROP TABLE TestLocks
CREATE TABLE TestLocks
(Id INT Identity (1,1) PRIMARY KEY CLUSTERED ,
Value CHAR(20) )

DECLARE @I INT
SET @I = 0
   WHILE 1=1
BEGIN
   INSERT INTO TestLocks(Value)
   VALUES ('Value' + CONVERT(VARCHAR(20), @I))
   SET @I = @I+1
   IF @I > 100
   BREAK;
END

Now we have learnt in earlier post Repeatable Read can cause phantom read.
Read Committed Isolation Level - Inconsistent analysis occurs when a second transaction accesses the same row several times and reads different data each time

In case of a Repeatble read for below query
BEGIN TRAN
SET TRANSACTION ISOLATION LEVEL REPEATABLE READ
SELECT * FROM TestLocks WHERE ID> 50
SELECT * FROM sys.dm_tran_locks where resource_type <> 'Database'
ROLLBACK TRAN

Lock request mode is Shared - Shared locks are placed and are held until the transaction completes

Same transaction when ran under SERIALIZABLE Isolation level would have a Range Lock.

BEGIN TRAN
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
SELECT * FROM TestLocks WHERE ID> 50
SELECT * FROM sys.dm_tran_locks where resource_type <> 'Database'

SERIALIZABLE - highest level, where transactions are completely isolated from one another. From MSDN "Range locks are placed in the range of key values that match the search conditions of each statement executed in a transaction. This blocks other transactions from updating or inserting any rows that would qualify for any of the statements executed by the current transaction"....

Hope it helps...Happy Reading...

SQL Tips

SQL Tip #1

PATINDEX - This function Returns the starting position of the first occurrence of a pattern in a specified
Example - Check to See String contains only numbers. Verify only numbers are present.

IF PATINDEX('%[a-z]%','11') > 0
    PRINT 'String'
ELSE
    PRINT 'Numbers'

SQL Tip #2
Query to Check for Available Physical Memory

SELECT
(Physical_memory_in_bytes/1024.0)/1024.0 AS PhysicalMemoryMb
FROM
sys.dm_os_sys_info

SQL Tip #3
ACID Properties short RECAP :)
Atomicity - Transaction is one unit of work. All or None. Either all of its data modifications are performed or none of them are performed
Consistency - Transaction must leave database in consistent state. Maintain data integrity. Governing Data Structures for Indexes/Storage must be in correct state. If a Transaction violates a constraint it must be failed.
Isolation - Keep Transaction Seperate. Concurrency is governed by Isolation levels.
Durability - Incase of System failures changes persisit and can be recovered on abnormal termination

SQL Tip #4
Difference between SET and SELECT

• You can assign one variable at a time using SET
• But assigning Multiples variables in one SELECT, SELECT is faster

User Stories....

Recently I am getting used to term "User Stories". I found this link useful. A good user story should also have a good acceptance test case with it. It need to have sufficient information. The objective of user stories is to connect the dots and identify and develop required functionality for end user. It's not that easy as I write as one liner :).

If you would like to have a good read on Bad or Good Agile, Read the link. Very good excerpts from the link. Take away from this article is

• Take things like unit testing, design documents and code reviews more seriously
• Bad Agile focuses on dates in the worst possible way: short cycles, quick deliverables, frequent estimates and re-estimates
• Encourage Peer Reviews and be open to respect others views and learn from peers

Having Domain and Technical workshop at the early phase of project would always help stakeholders and IT team understand requirements/functionality clearly.

Scrum Checklist
Kanban Vs SCRUM
Scrum-ban

Happy Reading.....

Deep Dive into Reverse Logistics

I registered myself to http://www.reverselogisticstrends.com/. You can access free downloads sections for good articles. Articles Reverse Logistics = Service Logistics, Reverse Logistics Checklist are very good reads.

Reverse Logistics Checklist provides clear directions while dealing with Customers
I am summarizing the checklist below
Prereturn - Clear Packaging Instructions, Warranty lookup from site, Warranty Registration directly based on customer registering the product, Self Test - Providing Diagnostic software for customer to test at this place
Return Request - Self ServiceRequest Creation, Request Acknowledgement through email, Call centre having visibility of stock available for replacement
Return Processing - Status lookup phone/website, Status update emails, Customer Survey
Having Joint Metrics defined with partners and tracking them on a timely manner would also help to focus on right areas of improvement.

If you want to know features of a Reverse Logistics Software refer the brochure for
BacTracs-Reverse Logistics Management System. This would give you good insight on Reverse Logistics implementation. Warranty Life Cycle Management

How many times my query has been executed ?

Thanks to Balmukund for his help. I wanted to know how many times a query has been executed from its query plan. Here is quick steps for this.
Step 1 - Create necessary tables for demo
use tempdb
CREATE TABLE DBO.TESTTable
(
    Id INT Identity(10,10),
    Name VARCHAR(20)
)

INSERT INTO DBO.TESTTable(Name)
Values ('Testvalue')
GO 50
Step 2 - Run below query
SELECT * FROM TESTTable WHERE Id = 20

Step 3 - I would like to know the plan for this query - Below DMV query would help

SELECT *
FROM sys.dm_exec_query_stats
CROSS APPLY sys.dm_exec_sql_text(plan_handle)
CROSS APPLY sys.dm_exec_query_plan(plan_handle)
WHERE TEXT LIKE '%TestTable%'

Step 4 - Fetch the Plan Handle from above step. Use the DMV sys.dm_exec_query_stats to find execution count of that plan based on plan handle
SELECT execution_count,* FROM sys.dm_exec_query_stats where plan_handle = 0x0600020078374D0040213585000000000000000000000000
 
This execution count would tell you how many times this plan has been executed. To check plan is cached  query cached_plans DMV can be used
SELECT * FROM sys.dm_exec_cached_plans where plan_handle = 0x0600020078374D0040213585000000000000000000000000
 
Hope it was useful. Have a Good Day.

Non-repeatable read and Phantom read

While trying to learn on concurrency problems I wanted to know difference between Non-repeatable read and Phantom read. The following post in MSDN is useful.

Nonrepeatable read: If somebody performed UPDATE or DELETE of any of the rows you read earlier.
Phantom: If anybody INSERTed a row within the range you had for an earlier query (i.e., you see new rows).

A Quick Experiment on Phantom Reads

Step 1 - Created a Test Table and Populated few records

USE TestDatabase
IF OBJECT_ID ('TestIsolation') IS NOT NULL DROP TABLE TestIsolation
CREATE TABLE TestIsolation
(Id INT Identity (1,1) PRIMARY KEY CLUSTERED ,
Value CHAR(20) )

DECLARE @I INT
SET @I = 0
WHILE 1=1
BEGIN
     INSERT INTO TestIsolation(Value)
     VALUES ('Value' + CONVERT(VARCHAR(20), @I))
     SET @I = @I+1
     IF @I > 100
     BREAK;
END

Step 2-Open a Session and Run Below Command

USE TestDatabase
GO
SET TRANSACTION ISOLATION LEVEL REPEATABLE READ
BEGIN TRAN
     SELECT * FROM TestIsolation
    WAITFOR DELAY '00:00:05'
    SELECT * FROM TestIsolation
ROLLBACK

Step 3 - Parallelly Open another session and run below command

USE TestDatabase
GO
BEGIN TRAN
     INSERT INTO TESTIsolation(VALUE)
     VALUES ('New Entry')
COMMIT

Step 4 - Output would be as below for Step 2. As you see newly inserted row is reflected in second select query (103 rows affected)

Note: For Step2 if You change isolation level to Serializable, You will see same number of affected rows  for the above case. A serializable scan acquires a key range lock which prevents the insertion of any new rows anywhere within the range

Another Quick Experiment on Nonrepeatable Reads

Nonrepeatable Read - Inconsistent analysis occurs when a second transaction accesses the same row several times and reads different data each time

Step 1 - Run the below query in one session

USE TestDatabase

GO
SET TRANSACTION ISOLATION LEVEL READ COMMITTED
BEGIN TRAN
     SELECT * FROM TestIsolation
     WAITFOR DELAY '00:00:05'
     SELECT * FROM TestIsolation
ROLLBACK

Step 2- When Step1 executes run Step2

USE TestDatabase
GO
BEGIN TRAN
    Update TESTIsolation
    SET VALUE = 'Modified Value'
    WHERE ID = 100
COMMIT

Step 3 - Output for Step 1

For Id 100 Data is modified as you can see above.

For Read-Committed Isolation level Advantages are

• Only committed changes are visible
• It acquires short lived share locks on a row by row basis
• The duration of these share locks is just long enough to read and process each row 

Disadvantages of Read-Committed Isolation Level

• Nonrepeatable Read - Inconsistent analysis occurs when a second transaction accesses the same row several times and reads different data each time
• Phantom reads occur when an insert action is performed against a row that belongs to a range of rows being read by a transaction.

Happy Learning!!!

SQL Tip for the Day

Tip 1 - Difference between Table Variables Vs Temporary Tables Vs Global Temporary Tables

Global Temporary Tables
1. Global temporary tables are available to all SQL Server connections.
2. All Sessions can access it
3. Same As Temp Tables Add ## before table name while declaring it

Table Variables
1. No Statistics associated with them
2. Do not participate in locking/transactions
3. Operations are not logged
4. Indexs cannot be created for table variables
5. They have scope associated only with current stored procedure it would be visible.
6. The only place where you can define columns to a table variable is in the declaration. ALTER TABLE not supported for Table Variable

Temp Tables
1. Statistics associated with them
2. Participate in locking/transactions
3. Operations are Logged
4. Add a single # before table name while declaring it
5. ALTER TABLE supported for Temp Tables
Reference - Link1, Link2, Link3, Link4, Good One - SangeethaShekar Blog
TempDB:: Table variable vs local temporary table


Tip 2 - When Clustered Index Columns are modified are Non-Clustured Indexes Re-built
Yes They are rebuilt. When Clustered Index column is re-ordered/arranged based on newly added/modified columns. Supporting no-clustered indexes would be as well rebuilt
Reference - Link1

Tip 3 - What resides in tempdb

• Inserted and deleted tables actually stored in tempdb
• User Created temp tables
• Online Index operation uses tempdb
• Worktables group by, order by (Ex- ORDER BY clause references columns not covered by any indexes, the relational engine may need to generate a worktable)
• MARS (Multiple Active Result Sets)
• In SQL 2005 Onwards When you use Row versioning data (Readcommitted, Snapshot Isolation level) is stored in tempdb

Happy Learning!!!

SQL Tip of the day

Tip 1
I need only Date part from date time. Different formats and examples

SELECT GETDATE()
SELECT CONVERT(char(20), GETDATE(), 101) --mm/dd/yyyy
SELECT CONVERT(char(20), GETDATE(), 1) --mm/dd/yy
SELECT CONVERT(char(20), GETDATE(), 0) --Month date yyyy hh:miAM (this is the default style)
SELECT CONVERT(char(20), GETDATE(), 1) --mm/dd/yy
SELECT CONVERT(char(20), GETDATE(), 112) --yyyymmdd
Tip 2
I want to group data based on month, year and day

SELECT GETDATE()
SELECT YEAR(GETDATE()),MONTH(GETDATE()), DAY(GETDATE())
Tip 3
I want to group data based on Hour, Minute and Second

SELECT GETDATE()
SELECT DATEPART(HOUR,GETDATE()), DATEPART(MINUTE,GETDATE()), DATEPART(SECOND,GETDATE())
Tip 4
COALESCE Function - Returns the first nonnull expression among its arguments

SELECT COALESCE(NULL+'Test',NULL,'Test1')--Test1
SELECT COALESCE('Test2'+'Test',NULL,'Test1')--Test2Test
SELECT COALESCE('Test2'+NULL,NULL,'Test1')--Test1
Tip 5
What is output for SELECT 'TEST'+ NULL
 
Cancatenation of anything to NULL will give only NULL output. Using ISNULL would fix it.
SELECT 'TEST'+ ISNULL(NULL,'')
Tip 6
Explore more on Datatype Date, Time, DateTime, SmallDateTime and Choose datatype based on need. This would help reduce space and appropriate use of datatypes.

IF OBJECT_ID ('TEST') IS NOT NULL DROP TABLE TEST
CREATE TABLE TEST
(
DateCol             Date,
TimeCol            Time,
DateTimeCol     DateTime,
SmallDTCol        SmallDateTime
)

INSERT INTO TEST (DateCol,TimeCol,DateTimeCol,SmallDTCol)
VALUES(GETDATE(), GETDATE(),GETDATE(),GETDATE())

SELECT * FROM TEST

What is Reverse Logistics.....

Reverse logistics has been defined as “... the term most often used to refer to the role of logistics in product returns, source reduction,recycling, materials substitution, reuse of materials,waste disposal, and refurbishing, repair and remanufacturing.” (Link)

A very good comparision on forward logistics vs reverse logistics presented below. Source Reverse Logistics Association.

Do find time to check Reverse Logistics Wiki
Reverse Logistics Framework as provided in Wiki. This kind of completely covers End-to-End Reverse Logistics Operations.

Very good white paper from UPS on ReverseLogistics
Key Learnings as Captured in paper as Summary

• Customer retention/satisfaction - Post Purchase Support for Repair is very important for better customer satisfaction
• Container reuse
• Recycling programs (Transport packaging)
• Damaged material returns
• Asset recovery/restock
• Downstream excess inventory (Seasonality)
• Hazardous material programs
• Obsolete equipment disposition
• Recalls

A good Example is also provided in the paper.
Possible options for reclaimed product

• Refurbish (Improve product beyond original specs)
• Recondition (Return product to original specs)
• Salvage (Separate components for reuse)
• Repair (Prepare for sale as a used product)
• Sell to 3rd Party
• Recycle
• Discard/Liquidation (Landfill)

The confluence of blockchain and IoT in Supply Chain can improve the efficiency of the operations, as per @EqualOcean's research. Link > https://t.co/5nWv9EIqNh @antgrasso via @LindaGrass0 #SupplyChain #blockchain #SmartContracts #IoT #Tech #DigitalTransformation pic.twitter.com/bDmPz7Njz0

— C-Suite Tech Point (@CsuiteTechPoint) May 25, 2020

Other Good Reads you may like
How to Develop A Reverse Logistics Strategy
Improve Your Business Applications
Advanced Exchange Service Model and the Secret of the ‘Black Hole’
Analysis of Reverse Logistics
Reverse Logistics Metrics (Customer Satisfaction, Financial Performance, Manufacturing (or Returns Processing and Refurbishment), Transportation and Warehousing)
Supply Chain Metrics

Biztalk Recipe No 4

Today we would look at publishing message to MessageBox and subscribing and transforming it.

1. Receive and Post Message M1 in MessageBox
2. Subscribe and Transform the message M1 into M2
3. Send the message M2 to another location

Step 1 - Create an Empty Biztalk Project
Step 2 - Create a New Message M1 Schema (ReceiveSchema)

Insert ChildFieldElement Id and Name











Step 3 - Create a New Message M2 Schema (Transform Schema)











Step 4 Add an Empty Biztalk Orchestration
STEP 5 Add Message1 Based on ReceiveSchema, Add Message2 Based on TransformSchema

Step 6 - Add a ReceiveShape, Set Activate-True, Message -> Message1

Step 7 - Specify Direct Binding for Port. Post Message in MessageBox

Step 8 - Add a Transform Shape, Under Properties of TransformShape ->Input Messages, SpecifySource Message_1 and Destination Message_2 and Click on OK

Step 9 - Biztalk Mapper would be launched. Map Id to Id & Name to Comments

Step 10 - Now go back to orchestration and add send shape and Assign Message_2 for Send Shape and bind port

Step 11 - Orchestration

Step 12 - Under Project->Biztalk Server Project -> Specify Application Name under Deployment Tab, Under Signing Tab sign the assembly

Step 13 Build and Deploy the application

Step 14 Start->Run-> btsmmc.msc
Step 15 - Goto Application. Create a Receive Location and a Send Port. Bind the orchestration and start it

Post a Message in Receive Location, It would be transformed and the message would be available in the SendPort Location Configured. Hope this article helps. Day by day would target to learn little by little.....Happy Learning!!!!

More Reads
BizTalk: Instance Subscription and Convoys: Details
BizTalk and SQL: Alternatives to the SQL receive adapter. Using Msmq to receive SQL data
BizTalk: Suspend shape and Convoy
BizTalk: Sample: Context routing and Throttling with orchestration