Results 1 to 2 of 2
  1. #1
    Join Date
    Aug 2015
    Posts
    1

    Challenge: Which Database model\Application should I use for this ?

    I am a scripter and I have basic knowledge about databases and its terms. Doing some research on global warming.
    I have a csv file delivered to me daily(ddmmyyyy.csv) which contains around 10 columns and 3000 rows. Already having 3500 such files 1 files keeps adding daily on working days.

    I am planning to create 2 databases out of these files because I have to perform some complex operations on it daily.

    First Here is the format of the csv file:
    ================================
    For date 01-August-2015
    --------------------------------------
    tablename=01082015.csv
    --------------------------------------
    Cityname,temperature,humidity,column4,col5,col6,co l7,col8,col9,col10
    mumbai,33,88,1234,1234,1234,1234,1234,1234,1234
    newyork,22,66,2345,2345,2345,1234,1234,1234,1234
    .
    .
    .
    3000 such records.
    ================================
    For date 02-August-2015
    --------------------------------------
    tablename=02082015.csv
    --------------------------------------
    Cityname,temperature,humidity,column4,col5,col6,co l7,col8,col9,col10
    mumbai,33,88,1234,1234,1234,1234,1234,1234,1234
    newyork,22,66,2345,2345,2345,1234,1234,1234,1234
    .
    .
    .
    3000 such records.
    ================================


    Database1 will simply contain the tables as table1 = yesterdays file as table, table2 = todays file as table ....
    Database1 will be simply the csv file converted to tables. So 1 table gets added daily to this database.

    Database2 -> Now since there are 3000 records in the csv file, there will be 3000 tables in database2
    Since 1 file is received daily, 1 record will be added to each of these 3000 tables daily.

    Database2 Example:
    =========================================
    table1name=Mumbai
    date,temperature,humidity,column4,col5,col6,col7,c ol8,col9,col10
    01-August-2015,33,88,1234,1234,1234,1234,1234,1234,1234
    02-August-2015,34,59,1234,1234,1234,1234,1234,1234,1234
    =========================================
    table2name=Newyork
    date,temperature,humidity,column4,col5,col6,col7,c ol8,col9,col10
    01-August-2015,33,88,1234,1234,1234,1234,1234,1234,1234
    02-August-2015,34,59,1234,1234,1234,1234,1234,1234,1234
    =========================================

    Things i'll be doing is checking average temperatues,humidity.. for a specific period etc..
    I have dedicated virtual machine for doing this with windows OS and 2GB of seperate ram for itself.
    By creating 2 databases I am just duplicating data of db1 into db2. I dont mind the time wasted in doing that bcas when I query I would like to have quick results.

    VIP Question:
    Which database s/w or type shoud I use for achieving this (like mssql, oracle, sqlite, mysql, msaccessdb etc.. ) ?
    Another question is, is my data model even correct ?

    Sorry I dont even know much terms about databses, please let me know If any more info is required. I have good knowledge about creating queries to achieve this and will willing to learn any new technology suggested. Thanks...

  2. #2
    Join Date
    May 2016
    Posts
    89
    Hi adiosped,

    could you upload your logical data model or even better conceptual data model ?

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •