If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Database Server Software > MySQL > 1 TB database size?

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 11-21-07, 10:49
ironCode ironCode is offline
Registered User
 
Join Date: Oct 2006
Posts: 12
1 TB database size?

Would it be safe to say that mysql can handle a TB database with acceptable performance? I mean queries under a minute. One table alone would reach roughly 400GB.

Is this reaching beyond mysql and needing to be moved over to something like oracle or can mysql with a cluster handle such a large database without taking 30 minutes per query?

I know query times really comes down to how you write the query but lets assume a average query for example.

Any thoughts?
Thanks
Reply With Quote
  #2 (permalink)  
Old 11-21-07, 15:08
mike_bike_kite mike_bike_kite is offline
vaguely human
 
Join Date: Jun 2007
Location: London
Posts: 2,519
I can't offer any tips but I am interested in what your trying to store.

What does the database hold and what's in the 400GB table? I assume it's time series data but it still seems like a lot of data. What type of reports are you intending on running - will it be on one day or one item or are you hoping to run a report across the whole database?

Mike
Reply With Quote
  #3 (permalink)  
Old 11-21-07, 15:16
ironCode ironCode is offline
Registered User
 
Join Date: Oct 2006
Posts: 12
It's telephone calls. We insert anywhere from 1.5 million to 6 million each day. so three months worth of data is roughly 130GB. We don't run any reports on this table directly. We build satellite tables which hold only the needed information for the report being called up. But these tables are populated from this much larger table holding the calls via cronjobs filing off php scripts. So there is a lot of back-end processing going on. It's this back-end processing that concerns me, as each of them and there are several that work directly with the larger table in the database.
Reply With Quote
  #4 (permalink)  
Old 11-22-07, 09:01
izeg izeg is offline
Registered User
 
Join Date: Nov 2007
Posts: 1
i have the same problem; 60000 row-data is logged per minute, it's okay while inserting but when data is requested for reporting, it takes too long.

a friend of mine advised me using distributed databases: there should be master and summary databases. the master db holds the latest data and the others store older data. the data stored in master database should be moved to other databases in a period of time/limited size. by doing it, the size of the db can be smaller and the searching can be easier.
Reply With Quote
  #5 (permalink)  
Old 12-04-07, 16:24
johndz johndz is offline
Registered User
 
Join Date: Dec 2004
Location: Broomfield, Colorado
Posts: 16
Not sure what version or engine you are using, but you may want to explore using a merge, or if you are up to 5.1 perhaps partitioning.

Are you using the myisam or innodb engine for the large table?
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On