Unanswered: Which data base is good for for storing 20gb a minute, 24x7..
Looking for your help in finding the right database for our application.
We have a real time calculation engine which generates 20 gb of data every minute, the application runs 24x7 and we need to keep last three days of data always.
Basically, the same set of calculation (1000 customers) happens every minute with little data change, hence we need to keep exiting calculations and new calculations as well. So, instead of over writing 20 gb of data every minute, we want to keep all calculations data.
I tried mongodb but the performance was very slow and finally thinking of storing in files in proto buf format and build a service to query the data. The query will be on customer id always.
Any guidance from you guys in solving this problem would of greate help.