"No one is harder on a talented person than the person themselves" - Linda Wilkinson ; "Trust your guts and don't follow the herd" ; "Validate direction not destination" ;
Showing posts with label SSIS. Show all posts
Showing posts with label SSIS. Show all posts

April 13, 2017

Day #64 - ETL for Data and Delta Data Management

Custom SSIS example sample for ETL setup for Data Extraction and Update

Scenario
  • Two Databases (Source and Target)
  • Example with Test Table with few columns
  • Ability to get New Data
  • Ability get Delta Data (Updates)
Step in SSIS Project

Step 1 - Create a Data Flow Task

Step 2 - Add connection managers for Source and Target Databases



Step 3 - The operators and layout is (Source Data -> Lookup in Target Database -> Insert / Update TargetDatabase)



Step 4 - OLEDB Data Source Settings


Step 5 -  Lookup to map for data



Step 6 - Lookup Mapping


Step 7 - Match Non-Matching for Insert / Updates



Step 8 - Match Destination Settings


Step 9 - Non Match Update Query



Step 10 - Non Match Update Params

Reference table script


SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[Table_1](
[Col1] [int] NULL,
[Col2] [int] NULL,
[Col3] [int] NULL
) ON [PRIMARY]
GO


Happy Learning!!!!

January 12, 2010

SSIS Error - [Execute SQL Task] Error: An error occurred while assigning a value to variable "xxxx"

I spent 2 hours for this. DataType was bigint and I was unable to assign output value. Below URL was useful to fix it http://blogs.msdn.com/mattm/archive/2007/04/18/why-can-t-i-store-my-bigint-result-in-an-int64-variable.aspx. Finally I have to type change base datatype to int and SSIS variable is INT32. This is on VS2005, SSIS 9.0

Second Learning is Eventlogging. Since, I am working on SSIS 2005 I tried for a VB code EventLog.Source Property

SQL Server 2005 Integration Services (SSIS): Custom Logging Using Event Handlers

SSRS Tip - Repeat Header / Keep Header Visible in Tables in RS 2008

More Reads
SSIS Balanced Data Distributor – Comparison