I did a quick search through the forum, however I couldn't find an answer to what I'm after, so I have a question for you all I hope this isn't the wrong section... I need to implement a free database system that is both fast and scalable. I anticipate for the data we will end up having, the databases will store in total, over 100 trillion rows of data.
There will basically be a bunch of servers in one rack generating data and inputting it into the database at ~10,000 records per second, then a website on the intranet allowing research staff to input a string, which the database will be searched for. So the only functions we really need are very simple(INSERT, UPDATE, SEARCH), however due to the amount of data I would anticipate we would need to spread the data across multiple databases and tables across multiple servers. At the moment we have about 6 x Dual Quad Core Servers put aside for database servers, however we are open to expansion if the need arises(no doubt it will). The searching needs to take less than 20 minutes per query, with up to 10 queries happening simultaneously.
So can anyone recommend a good free database system? Or is that not really possible? I'm thinking MySQL or PostgreSQL as they are the only two I have dealt with apart from MSSQL.
Also, it would be best if it ran on linux....
Last edited by markosolo; 09-26-10 at 03:04.
How many concurrent users are we talking about here?
How many fields, and of what kind, are to be searched for?
Can you give up a better idea as to the kind of data you're discussing? And, its structure?
What kind of indexing in anticipated? You mention text data.
"Lisa, in this house, we obey the laws of thermodynamics!" - Homer Simpson
"I have my standards. They may be low, but I have them!" - Bette Middler
"It's a book about a Spanish guy named Manual. You should read it." - Dilbert