If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Database Server Software > DB2 > How to Efficiently Insert 20K records at a time??

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 05-23-06, 16:51
dhani dhani is offline
Registered User
 
Join Date: May 2006
Posts: 3
How to Efficiently Insert 20K records at a time??

Hi All,
Currently we are using hibernate to perform insert, which is taking approximately 80 Secs to insert 20K records at a time. I want to reduce this as fast as possible. The table I am inserting has 15 columns. Please advise me how do I increase the performance of this insert.
Thank You in advance.
Dhani
Reply With Quote
  #2 (permalink)  
Old 05-23-06, 17:06
Marcus_A Marcus_A is offline
Registered User
 
Join Date: May 2003
Location: USA
Posts: 5,196
If you can't use a LOAD or IMPORT command (LOAD is faster) then try using this syntax to insert faster:

INSERT INTO DEPARTMENT (DEPTNO, DEPTNAME, ADMRDEPT)
VALUES
(’B11’, ’PURCHASING’, ’B01’),
(’E41’, ’DATABASE ADMINISTRATION’, ’E01’)
etc.
etc.

Insert about 100 rows at once and do a commit, then next 100, etc.

Make the following changes to the db configuration (db2 get db cfg):

Increase the logbufsz to 256.
Increase dbheap by 256 (logbufsz comes out of dbheap).
Increase the num_iocleaners at least 4.
set chngpgs_thresh to 10 (no lower than 20).
set logfilesz to at least 10000.
__________________
M. A. Feldman
IBM Certified DBA on DB2 for Linux, UNIX, and Windows
IBM Certified DBA on DB2 for z/OS and OS/390
Reply With Quote
  #3 (permalink)  
Old 05-24-06, 09:30
dhani dhani is offline
Registered User
 
Join Date: May 2006
Posts: 3
Thank you for your suggestion. When I was searching in google, I found out that I should not use static insert stm but I should use parameterized statement to increase the performance. Also I can not change any of the DB configuration because it will impact the other queries. Is it possible to change these parameters through JDBC just for this query??.
Please let me know your suggestion.
Thank You
Reply With Quote
  #4 (permalink)  
Old 05-25-06, 09:53
Brett Kaiser Brett Kaiser is offline
Window Washer
 
Join Date: Nov 2002
Location: Jersey
Posts: 10,303
Why can't you use LOAD RESUME?
__________________
Brett
8-)

It's a Great Day for America everybody!

dbforums Yak CorralRadio 'Rita
dbForums Member List
I'm Good Once as I ever was

The physical order of data in a database has no meaning.
Reply With Quote
  #5 (permalink)  
Old 05-25-06, 09:57
dhani dhani is offline
Registered User
 
Join Date: May 2006
Posts: 3
Data is generated in the Application, Can I use load to store the data from J2ee application to DB??. I guess not.. Please confirm..
Reply With Quote
  #6 (permalink)  
Old 05-25-06, 11:09
Brett Kaiser Brett Kaiser is offline
Window Washer
 
Join Date: Nov 2002
Location: Jersey
Posts: 10,303
What do you mean by generated?

Is this a batch process that spits out 20k rows? Or is it OLTP?

If it is batch, I would write it to a file, then I would load....but then again, I'm not an application grunt...ok, so I was, but why aren't you using stored procedures...I mean the data must be coming from somewhere
__________________
Brett
8-)

It's a Great Day for America everybody!

dbforums Yak CorralRadio 'Rita
dbForums Member List
I'm Good Once as I ever was

The physical order of data in a database has no meaning.
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On