If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Database Server Software > Informix > Loading into table is very slow !!

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 10-22-03, 12:25
pd12 pd12 is offline
Registered User
 
Join Date: Sep 2003
Posts: 22
Loading into table is very slow !!

Hello All,

I am trying to load 34mb file into table and it's very slow.
I thought it was b'coz of fragmentation, even after removing fragmentation it's very slow.
How can i make it fast, we have to load the huge data in many tables for testing purpose.

Suggets me.

Thanks,
PD
Reply With Quote
  #2 (permalink)  
Old 10-22-03, 13:11
sundialsvcs sundialsvcs is offline
Registered User
 
Join Date: Oct 2003
Posts: 706
Re: Loading into table is very slow !!

Quote:

I am trying to load 34mb file into table and it's very slow.
I thought it was b'coz of fragmentation, even after removing fragmentation it's very slow.
How can i make it fast, we have to load the huge data in many tables for testing purpose.

34 megabytes is not a lot of data. If you're experiencing bad performance loading it, try disabling or dropping all of the indexes .. then load the data .. then turn the indexes back on.

When a database "rebuilds an index from scratch," it can do so efficiently by sorting the data and building the index from the sorted data. Very fast. What is extremely slow is doing 34 megabytes' worth of index updates, one at a time.

It's also a good idea to do the update as a series of small transactions, say every 1000 records or so, so that the computer doesn't plan to be able to "roll back" 34 megs' worth of data... a process that would produce a huge rollback file for absolutely no good reason.
__________________
ChimneySweep(R): fast, automatic
table repair at a click of the
mouse! http://www.sundialservices.com
Reply With Quote
  #3 (permalink)  
Old 10-22-03, 16:46
pd12 pd12 is offline
Registered User
 
Join Date: Sep 2003
Posts: 22
Hello Sundialsvcs,

Thank you.
I guess i have under-estimated the sizes of data files i am loading.
It's 421463718 bytes.

It is a datamart kind of database and doesn't have any index and any constraints for that matter.
Thought External table would be a good idea and it happened so very fast with the same for loading. But when i trying to load the data with joins again it's slowing down the complete process.

Now, i don't have nay solution for the same.
Yes, i tried with spliting files in several files and it went thru' faster than with one file.


Regards,
PD
Reply With Quote
  #4 (permalink)  
Old 10-23-03, 14:29
Roelwe Roelwe is offline
Registered User
 
Join Date: Aug 2002
Location: Belgium
Posts: 534
Have you tried the High Performance Loader?
__________________
rws
Reply With Quote
  #5 (permalink)  
Old 10-23-03, 16:40
pd12 pd12 is offline
Registered User
 
Join Date: Sep 2003
Posts: 22
Hello rws,

Thank you.
I am afraid i have no idea about High Performance loader, where can i find the info about the same?

I will give a try when i read/know more about it.

Regards,
PD
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On