Results 1 to 2 of 2
  1. #1
    Join Date
    Jul 2017
    Posts
    2

    50k-80k records DB - looking for suggestions

    Hi there

    First off, I'm no DB expert, I've used a number of DBs as a user, writing queries, etc, my day job is a BA so I've really only written requirements, worked with DB developers to understand how their solution can solve a particular business problem, etc, so have always relied on experts to do the actual DB planning and build, etc. I've only ever created a few small DBs in Access many years ago at uni, so am definitely at bit of a disadvantage here.

    This project is non-work related, I've come across a bunch of publically-available historic records which date back to the late 1800s and this data would be extremely valuable for local schools and universities, but the way the data is currently stored is basically unusable. So after pondering on it for a few months I decided to see whether I can build something that the public can use, which won't cost me an arm and a leg, and that I can complete myself say over the next year or two, dedicating about 1-2 hours per night after work (basically going to stop playing ARK Survival and do something useful instead).

    The data:
    There are 110 html files (with minimal formatting, each is between 6 and 18Mb) which contain text, tables, etc. My thoughts are that I'll have to manually move this data in as I locate the necessary tables, etc. I'll take proper stock of all types of data and will plan the DB to understand the total number of tables, their relationship, etc. So far I've identified approximately 260-300 tables just to give me an idea of the scale of this. Each table carries between 50 and 500 records

    The objective:
    I'd like to make it a free-to-access public DB where anyone can make sense of the available information. My assumption is that I'll need to build a lot of forms specific to each type of data. So based on that I assume I'll need to build a couple of hundred forms once all the data is in. This objective is pretty vague but without going through the data in extreme detail this is all I have at this point.

    Based on the above, I was wondering if anyone can recommend a cloud DB solution which:
    - can handle the amount of data/tables/forms that I mentioned
    - is affordable (this is relative, i guess I can't pay enterprise rates, but a small monthly fee is no problem)
    - has good documentation (as I'll more than likely have to learn a lot of this from scratch)

    Any help would be greatly appreciated.

  2. #2
    Join Date
    Jul 2017
    Posts
    2
    After doing some more research I decided to:
    - install MySQL on a virtual server
    - install MySQL Workbench
    - install front end of some sort, looking at v.front but looks like there are a few options there

    then once I got it all to work, will start on planning the db, etc. and if all goes well, then I'll be able to migrate this to an external host.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •