Results 1 to 5 of 5
  1. #1
    Join Date
    Oct 2006
    Posts
    12

    Unanswered: 1 TB database size?

    Would it be safe to say that mysql can handle a TB database with acceptable performance? I mean queries under a minute. One table alone would reach roughly 400GB.

    Is this reaching beyond mysql and needing to be moved over to something like oracle or can mysql with a cluster handle such a large database without taking 30 minutes per query?

    I know query times really comes down to how you write the query but lets assume a average query for example.

    Any thoughts?
    Thanks

  2. #2
    Join Date
    Jun 2007
    Location
    London
    Posts
    2,527
    I can't offer any tips but I am interested in what your trying to store.

    What does the database hold and what's in the 400GB table? I assume it's time series data but it still seems like a lot of data. What type of reports are you intending on running - will it be on one day or one item or are you hoping to run a report across the whole database?

    Mike

  3. #3
    Join Date
    Oct 2006
    Posts
    12
    It's telephone calls. We insert anywhere from 1.5 million to 6 million each day. so three months worth of data is roughly 130GB. We don't run any reports on this table directly. We build satellite tables which hold only the needed information for the report being called up. But these tables are populated from this much larger table holding the calls via cronjobs filing off php scripts. So there is a lot of back-end processing going on. It's this back-end processing that concerns me, as each of them and there are several that work directly with the larger table in the database.

  4. #4
    Join Date
    Nov 2007
    Posts
    1
    i have the same problem; 60000 row-data is logged per minute, it's okay while inserting but when data is requested for reporting, it takes too long.

    a friend of mine advised me using distributed databases: there should be master and summary databases. the master db holds the latest data and the others store older data. the data stored in master database should be moved to other databases in a period of time/limited size. by doing it, the size of the db can be smaller and the searching can be easier.

  5. #5
    Join Date
    Dec 2004
    Location
    Broomfield, Colorado
    Posts
    16
    Not sure what version or engine you are using, but you may want to explore using a merge, or if you are up to 5.1 perhaps partitioning.

    Are you using the myisam or innodb engine for the large table?

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •