If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Database Server Software > MySQL > insert huge # of rows

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 08-04-04, 06:28
TimKaine TimKaine is offline
Registered User
 
Join Date: Aug 2004
Posts: 2
insert huge # of rows

hi
newbie here. Whenever I try to insert huge number of rows (say 25000 records) into a table my transaction timesout. I use this query

insert into gTable select A,B, C, D, E,F,G,sum(V) ,((Floor(H/10000))*10000) as ttime, min(V) , max(V), sum(V)/6 from preTable where (A <=25000 and A>0) group by B,C,D,E,F,G,H

Few columns in this table(innodb type) are indexed and few have FK constraints. This query takes anything between 10 min to 20 min to complete. Removing FKs does speed up insert time but stillam running into similar issues. any help?
Reply With Quote
  #2 (permalink)  
Old 08-07-04, 13:56
bstjean bstjean is offline
Registered User
 
Join Date: Sep 2002
Location: Montreal, Canada
Posts: 219
Quote:
Originally Posted by TimKaine
hi
newbie here. Whenever I try to insert huge number of rows (say 25000 records) into a table my transaction timesout. I use this query

insert into gTable select A,B, C, D, E,F,G,sum(V) ,((Floor(H/10000))*10000) as ttime, min(V) , max(V), sum(V)/6 from preTable where (A <=25000 and A>0) group by B,C,D,E,F,G,H

Few columns in this table(innodb type) are indexed and few have FK constraints. This query takes anything between 10 min to 20 min to complete. Removing FKs does speed up insert time but stillam running into similar issues. any help?
What takes the most time here? Did you time the SELECT to see how much time it took to execute? Did you try an EXPLAIN on your select to verify if your query wouldn't benefit from an index on column A?
Reply With Quote
  #3 (permalink)  
Old 08-13-04, 11:08
TimKaine TimKaine is offline
Registered User
 
Join Date: Aug 2004
Posts: 2
insert huge # of rows

--"What takes the most time here? Did you time the SELECT to see how much time it took to execute? Did you try an EXPLAIN on your select to verify if your query wouldn't benefit from an index on column A?"

The select part of the query does not take time. It is the inserts ..and it slows down as the table grows. Here is the result of the explain query.

*************************** 1. row *************************
table: preTable
type: range
possible_keys: PRIMARY
key: PRIMARY
key_len: 8
ref: NULL
rows: 50408
Extra: Using where; Using temporary; Using filesort
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On