Use of Script Task to create SQL table using VB.Net in SSIS

The Below Code will be useful in the cases when we need to deal with flat files, wherein the file structure may be dynamic and we are dealing with multiple files each one with a different schema definition.

The approach will be here is to loop through each file and then read the first line of each file and read the column names from the file and then dynamically create the table and then load the data.

The below code is used to read the column information from the file and then generate a SQL Command which will create the table in the database.

Here it goes - 

Code to dynamically create a table using the Input File Data –

Public Sub Main()
        Dim objReader As New StreamReader(Dts.Variables("SRCDATAFILE").Value.ToString)
        Dim sLine As String = ""
        Dim arrText As New ArrayList()
        sLine = objReader.ReadLine()
        'MsgBox(sLine)
        sLine = sLine.Replace(ControlChars.Tab, "|")
        'MsgBox(sLine)
        sLine = Trim(sLine.Replace(" |", "|").Replace("| ", "|"))
  sLine = "IF EXISTS(SELECT 1 FROM INFORMATION_SCHEMA.TABLES WHERE     TABLE_NAME = My_Table') DROP TABLE RBI_Product_stage_tmp; Create Table My_Table ([" & Replace(sLine, "|", "] Varchar(150), [") & "] Varchar(150))"
        'MsgBox(sLine)
        Dts.Variables("CREATESQLCMD").Value = sLine
        If CBool(Dts.Variables("I").Value) Then
            Dts.Variables("CREATESQLCMD").Value = ""
        End If
        objReader.Close()

        Dts.TaskResult = ScriptResults.Success
End Sub

Happy Learning !!

Comments

Post a Comment

Popular posts from this blog

Deleting Duplicate Data From SQL Server Table

Duplicate data is a very well known and frequently occurring issue in any DBMS and dealing with deletion of duplicate data is something not that easy for one specially one who is dealing with crucial and huge data. There are 2 scenarios which can come while deleting duplicate data - 1. when we have an identity column in our table containing duplicates. The problem becomes much easier here and we can just keep the data with MIN or MAX keys removing all other. --Create New Table create table Person ( ID int identity ( 1 , 1 ), Name varchar ( 30 ), age int ) --Insert Data into Table with multiple Duplicate entries insert into Person ( Name , age ) values ( 'Ashish' , 24 ) insert into Person ( Name , age ) values ( 'Ashish' , 24 ) insert into Person ( Name , age ) values ( 'Gaurav' , 26 ) insert into Person ( Name , age ) values ( 'Nayan' , 25 ) insert into Person ( Name , age ) values ( 'Nayan...
Read more

SSIS vs. T-SQL

This is a common question which comes often before any developer to choose between the two as to follow which path to choose as SSIS or T-SQL for developing ETL solutions. well i also got similar scenario some time back and was in a dilemma to decide this. on a big picture it's always better to go with conventional way i.e. using T-SQL and not to go with the tool. But on doing some more reading and research over many blogs and forums and also the development phase of my work over some 2 and a half odd years. I found some good reasons and explanations over this as not to always stick to a particular approach and do proper analysis of the requirements to take a decision. Below are some of the key points which i was able to collect regarding this. 1.  If the data does need to be persisted temporarily it can be to a raw file in SSIS whereas T-SQL requires temporary tables for which there may be a management/security overhead. 2.  SSIS have a very good visual rep...
Read more

Excel Based Reporting

Hello Everyone, so today my focus is on to give insights about how can we deal with scenarios where the client wants us to help them with Excel based reporting. We will be doing this using SSIS as we will need automation of the report on a daily basis. Technology Items used - 1. SSIS 2. Excel files (format files) 3. Macro codes 4. Conditional formatting. 5. SSIS 6. C#.Net / VB.Net Script Components Step-1:  - We will be writing a few macro codes to do the same as we will need the numeric columns to be in the numeric while we insert the data from SSIS it will automatically convert it to Text. Macro-1 - Code to Convert Text to Integer. Sub Function_Name() sheetlist = Array("Sheet1", "Sheet2", "Sheet3", "Sheet4", "Sheet5", "Sheet6", "Sheet7") For i = LBound (sheetlist) To UBound (sheetlist) Worksheets(sheetlist(i)).Activate ' Your Code Goes Here Range("F:F").Select 'specify the r...
Read more