If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Database Server Software > MySQL > Insert 30 millions rows

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 06-07-06, 05:20
holy_joy holy_joy is offline
Registered User
 
Join Date: May 2006
Posts: 11
Insert 30 millions rows

Hello,
I want to move or backup 30 millions rows from a MySQL database to another one (also MySQL).
Please give me some advices on how can I do this process in the fastest way AND especially I need to know the required time to do this process. Thanks.
Reply With Quote
  #2 (permalink)  
Old 06-07-06, 05:22
holy_joy holy_joy is offline
Registered User
 
Join Date: May 2006
Posts: 11
Insert 30 millions rows

Hello,
I want to move or backup 30 millions rows from a database to another one.
Please give me some advices on how can I do this process in the fastest way AND especially I need to know the required time to do this process. Thanks.
Reply With Quote
  #3 (permalink)  
Old 06-07-06, 05:23
holy_joy holy_joy is offline
Registered User
 
Join Date: May 2006
Posts: 11
Insert 30 millions rows

Hello,
I want to move or backup 30 millions rows from a database to another one.
Please give me some advices on how can I do this process in the fastest way AND especially I need to know the required time to do this process. Thanks.
Reply With Quote
  #4 (permalink)  
Old 06-07-06, 05:45
baburajv baburajv is offline
Registered User
 
Join Date: Feb 2004
Location: Bangalore, India
Posts: 242
if you are planning to move data from a table in one db to another, try bcp..
__________________
Cheers....

baburajv
Reply With Quote
  #5 (permalink)  
Old 06-07-06, 06:08
sathyaram_s sathyaram_s is offline
Super Moderator
 
Join Date: Aug 2001
Location: UK
Posts: 4,538
since it involves two different databases, I would think the best way is to export the data to a file from the source and LOAD into the Target.

At the source :
EXPORT TO table1.ixf of ixf select * from table1

On the target:
LOAD from table1.ixf of ixf savecount 1000000 insert into table1

It is not easy to predict the time that will be needed. It will depend on a number of factors like the table and index defn, row size, your disk system, db and dbm config etc.

HTH

Sathyaram
__________________
Visit the new-look IDUG Website , register to gain access to the excellent content.
Reply With Quote
  #6 (permalink)  
Old 06-07-06, 07:42
Thrasymachus Thrasymachus is offline
SQL Server Street Fighter
 
Join Date: Nov 2004
Location: Down The Rabbit Hole
Posts: 7,979
are you moving the whole database or just part of it?
__________________
software development is where smart people go to waste their lives
Reply With Quote
  #7 (permalink)  
Old 06-07-06, 09:06
blindman blindman is offline
World Class Flame Warrior
 
Join Date: Jun 2003
Location: Ohio
Posts: 11,732
Nobody can tell you the required time. You will have to try it and see, and it will probably vary some from one execution to the next as network activity, CPU usage, and drive space change.
__________________
If it's not practically useful, then it's practically useless.

blindman
www.chess.com: "sqlblindman"
Reply With Quote
  #8 (permalink)  
Old 06-07-06, 09:22
Brett Kaiser Brett Kaiser is offline
Window Washer
 
Join Date: Nov 2002
Location: Jersey
Posts: 10,303
Quote:
Originally Posted by holy_joy
in the fastest way
Do we get to rearchitect the system as well?

Is this one table? Is the database on another server (I would hope so).

what's the color of the hair of the women sitting closest to you?
__________________
Brett
8-)

It's a Great Day for America everybody!

dbforums Yak CorralRadio 'Rita
dbForums Member List
I'm Good Once as I ever was

The physical order of data in a database has no meaning.
Reply With Quote
  #9 (permalink)  
Old 06-07-06, 09:43
snorp snorp is offline
Registered User
 
Join Date: Apr 2004
Location: Europe->Sweden->Stockholm
Posts: 71
Quote:
Originally Posted by holy_joy
Hello,
I want to move or backup 30 millions rows from a MySQL database to another one (also MySQL).
Please give me some advices on how can I do this process in the fastest way AND especially I need to know the required time to do this process. Thanks.
What is the table type? With MyISAM you can copy the files directly, but you don't want to do that with MySQL running.

How long it takes? Well, it depends on your amount of memory, number and speed of CPUs, version of MySQL, your operating systems, number and speed of disks, buses, connections to SAN or fileservers, ...

I would use mysqldump to dump table data from server A directly to server B. Just redirect the output to the MySQL client connected to your target server and go grab some lunch. With reasonable equipment, you should probably be able to expect 10,000 rows/second, so it might take an hour or so.
Reply With Quote
  #10 (permalink)  
Old 06-07-06, 09:47
blindman blindman is offline
World Class Flame Warrior
 
Join Date: Jun 2003
Location: Ohio
Posts: 11,732
What next? Are you going to ask whether the carpet matches the drapes?

(In the office, of course...)
__________________
If it's not practically useful, then it's practically useless.

blindman
www.chess.com: "sqlblindman"
Reply With Quote
  #11 (permalink)  
Old 06-07-06, 09:59
Pat Phelan Pat Phelan is offline
Resident Curmudgeon
 
Join Date: Feb 2004
Location: In front of the computer
Posts: 12,609
Ok, I'll byte. Holy_Joy must be doing an assignment that requires them to find the best database for this problem. So far they've asked for DB2, Microsoft SQL, and MySQL. The teacher must have done the legwork for Oracle as an example for the class!

-PatP
Reply With Quote
  #12 (permalink)  
Old 06-07-06, 09:59
Pat Phelan Pat Phelan is offline
Resident Curmudgeon
 
Join Date: Feb 2004
Location: In front of the computer
Posts: 12,609
Ok, I'll byte. Holy_Joy must be doing an assignment that requires them to find the best database for this problem. So far they've asked for DB2, Microsoft SQL, and MySQL. The teacher must have done the legwork for Oracle as an example for the class!

-PatP
Reply With Quote
  #13 (permalink)  
Old 06-07-06, 09:59
Pat Phelan Pat Phelan is offline
Resident Curmudgeon
 
Join Date: Feb 2004
Location: In front of the computer
Posts: 12,609
Ok, I'll byte. Holy_Joy must be doing an assignment that requires them to find the best database for this problem. So far they've asked for DB2, Microsoft SQL, and MySQL. The teacher must have done the legwork for Oracle as an example for the class!

-PatP
Reply With Quote
  #14 (permalink)  
Old 06-07-06, 10:10
snorp snorp is offline
Registered User
 
Join Date: Apr 2004
Location: Europe->Sweden->Stockholm
Posts: 71
Aha, interesting. But what kind of assignment is that? Seems pretty useless to me
Reply With Quote
  #15 (permalink)  
Old 06-08-06, 01:24
holy_joy holy_joy is offline
Registered User
 
Join Date: May 2006
Posts: 11
lol PAT PHELAN!
I hope you byted... But I don't think u have an idea of the problem here, maybe in few month u'll probably guess
Anyway thanks for replying!!!!!!!!! it was funny
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On