Results 1 to 7 of 7
  1. #1
    Join Date
    Dec 2004
    Posts
    47

    Unanswered: Batch insert (OLAP)

    Hi can any one help me with the skeleton script (sample one)of running Bulk insert in batches........ I need to do it in batches as the input data is huge.....

    The logic is I have to insert thru bcp in fact table...

    After that batch execution for 50,000 thousand record.... wise.... if any of the batch failes i need to identify and have to rerun from that point onwards...... this is OLAP thing...

  2. #2
    Join Date
    Nov 2002
    Location
    Jersey
    Posts
    10,322
    what's the data source?

    50,000 ain't that much btw
    Brett
    8-)

    It's a Great Day for America everybody!

    dbforums Yak CorralRadio 'Rita
    dbForums Member List
    I'm Good Once as I ever was

    The physical order of data in a database has no meaning.

  3. #3
    Join Date
    Dec 2004
    Posts
    47
    50,000 record comes in one batch that waht I meant....... so if there is 1 million record 20 batches will be there....

    The source come from DB2 or so which we get it as source file.... we create staging table for that in Sql Server.... Now we need to do the rest porting data to Fact & Dimension tables

  4. #4
    Join Date
    Feb 2004
    Location
    San Antonio, TX
    Posts
    565
    what script?
    you can use create a failover process in dts to allow x number of error rows to pass through the ETL and then you can clean them up the next day if you prefer. its part of the error reporting process in dts tasks.

  5. #5
    Join Date
    Nov 2002
    Location
    Jersey
    Posts
    10,322
    So the data is already in a table and you're trying to limit the impact to the logs...can you describe your process how are you building your warehouse..
    Brett
    8-)

    It's a Great Day for America everybody!

    dbforums Yak CorralRadio 'Rita
    dbForums Member List
    I'm Good Once as I ever was

    The physical order of data in a database has no meaning.

  6. #6
    Join Date
    Dec 2004
    Posts
    47
    I didnt get u I never asked for error reporting stuff!!!!!

  7. #7
    Join Date
    Dec 2004
    Posts
    47
    http://www.winnetmag.com/article/art...903/42903.html

    I was trying to do the same way how the above article describe to solve such kinda stuff

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •