If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Database Server Software > DB2 > [Query: Technical] DB2 9.1 bulk load

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 05-06-10, 09:28
changusee2k changusee2k is offline
Registered User
 
Join Date: May 2010
Posts: 2
Cool [Query: Technical] DB2 9.1 bulk load

Hi All,

We have a project requirement to upload around 54 million records (available in the form of a csv file) into a DB2 table.
The destination table is empty, has 37 columns (all simple data types – no LOBs) and has no column constraints or composite keys.

We tried this activity with the import/load command, but it took us around 6 hrs to load 4 million records! DB2 Data studio did not help us as well.

Can someone please let us know of any easier/faster method to bulk load such a large volume of data in DB2.

Thanks
Chandra.
Reply With Quote
  #2 (permalink)  
Old 05-06-10, 09:36
dbzTHEdinosaur dbzTHEdinosaur is offline
Registered User
 
Join Date: Jun 2007
Location: germany
Posts: 96
buy more iron.
__________________
Dick Brenholtz, Ami in Deutschland
Reply With Quote
  #3 (permalink)  
Old 05-06-10, 09:41
changusee2k changusee2k is offline
Registered User
 
Join Date: May 2010
Posts: 2
[Query: Technical] DB2 9.1 bulk load

Mate,

u got to be kidding me to have posted such a humorous reply..

jokes apart..

can u pls provide me with some really technical solution, as we need to find a way to get this Bulk load thing completed at the earliest?

Cheers
Chandra.
Reply With Quote
  #4 (permalink)  
Old 05-06-10, 09:55
dbzTHEdinosaur dbzTHEdinosaur is offline
Registered User
 
Join Date: Jun 2007
Location: germany
Posts: 96
what is the 'purest' input format for data to the load utility?

a csv file will work,
but does the utility have to perform some kind of 'reformatting'
before the actual loading to the tables takes place?

apparently this is a 'one-off' exercise.
either bite the bullet and do the load as is
or
'normalize' your csv file to a type that requires no formatting by the utility - which will also take time - but may make the load process run faster.
__________________
Dick Brenholtz, Ami in Deutschland
Reply With Quote
  #5 (permalink)  
Old 05-06-10, 10:26
Cougar8000 Cougar8000 is offline
Registered User
 
Join Date: Nov 2005
Location: IL
Posts: 554
Are you sure you are using LOAD? I have never seen it take that long to load that few records. Post your command.
__________________
--
IBM Certified DBA on DB2 for Linux, UNIX, and Windows

DB2 v9.1.0.2 os 5.3.0.0
Reply With Quote
  #6 (permalink)  
Old 05-06-10, 13:41
dr_te_z dr_te_z is offline
Registered User
 
Join Date: Jan 2009
Location: Zoetermeer, Holland
Posts: 555
Look at the disk-layout and tablespaces. No SMS I hope? No raid5 I hope? You need the best write-performance and DMS tablespaces already in the right size and spread over as many disks as possible. And "util_heap_sz " max.size?
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On