Results 1 to 4 of 4
  1. #1
    Join Date
    Mar 2013
    Posts
    2

    Unanswered: Quick Question about a very large DB

    I'm looking at pulling some historical option data into a database. I'm looking at about 200GBs worth of data that would be dropped into 3 tables. The main index for all tables will be a symbol (string) and a date.

    First question is, if I use MySQL, will the size of the data present any problems if I want to look up the latest quote on a given symbol?

    Is it better to separate the historical data from the most recent data for lookup performance? I'm not too concerned if the historical data takes a little longer to fetch, but I would like the most recent data to be fairly quick.

    Thanks

  2. #2
    Join Date
    Feb 2004
    Location
    In front of the computer
    Posts
    15,579
    Provided Answers: 54
    With only 200 Gbs of data, MySQL shouldn't have any problem handling the data volume. With good schema design and supporting indexes, almost all of the queries ought to have sub-second response time.

    Since this is a MySQL specific question, would you like me to move it to the MySQL forum for you?

    -PatP
    In theory, theory and practice are identical. In practice, theory and practice are unrelated.

  3. #3
    Join Date
    Mar 2013
    Posts
    2
    Sure. Move it. But I was really asking in general.

    I'm not a dB guy. 200GB seemed like a lot to me since it took my machine 24 hours to crunch. Shows what I know

    Thanks

  4. #4
    Join Date
    Feb 2004
    Location
    In front of the computer
    Posts
    15,579
    Provided Answers: 54
    More than anything you need to ensure that your database schema is normalized, and that you have an appropriate number of indexes to help your queries along. This isn't that hard to do, but it can be daunting for a first time database user. At 200 Gb, that doesn't need to make a server sweat if you do a good job planning and preparing for it.

    -PatP
    In theory, theory and practice are identical. In practice, theory and practice are unrelated.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •