Unanswered: MySql with Full text search massive use and huge database
what i intend to do is to create a web application that needs to work with large capacity of inforamtion (around the 2-3 Terra) most of it in text, so people will have to search inside this huge data storage, and the big problem is let to estimated amount of 30,000 users work in this system in the rang of 10 min which can lead to more then 100-200 conccurent users on the system.
i want very much use mysql with the full text search options which made my life much easiar, and i known the fact and needs of using more then one (even 2 or 3) app server for web but what the hell can i do to minimize the scattering of the database server to minimum computers (i would like it to be one, with copies for balancing) but still that it will be able to handle the traffic of many users in the same time and many searches in the same time.
things that i haved thought that may be a problem are:
to devide the database to more then one sever but still dont harm the full text search
to work with more then half terra with one machine, is it good or its better to devide it to more then one and then what to do to connect between them with out too much complications
if you can help me decide or give advices concernig the fulltext search working with large database it will be greate
i know that wikipedia working in the same way with them large database. PHP, MySql with fulltext search on one system with many copies of it.
Running more than one MySQL for load balancing is very well possible. But don't mix it up with 'deviding' the database ! That's something completely differerent. I think what you are looking for is running 2 or 3 MySQL servers, each with their own complete database. Not running a number of MySQL servers, each with a small part of the one database ! You might even think of running multiple Webservers using some kind of load balancing mechanism and connect each webser with it's own MySQL server.
hope this helps a little ...
> SELECT * FROM users WHERE clue > 0;
Empty set (0.00 sec)
and about your solution i known that i probebly need to make a few mirros of the same database in the same network to serve the load traffic of the system, but what i am not sure about is how to put huge database maybe more then 1.5Tb on one/few machine with indexe by mysql fulltext search that need good process abilities is it possible?
And if not, what else can i do?