I couldn't find an answer anywhere on the site, if there is one, could you please let me know where to go? Thanks.
Here goes. We receive large csv files, around a million records a month. What we need to do is:
- Of course, import these records
- Be able to run failry heavy reports on around 3 months worth of data (about 3 million records)
- Wanting to store at least a years worth of data
What I need to know:
- What server(s) spec (CPU, HDD, memory, etc) would we need to import and run reports quite fast. Don't want the reports/imports to take days to run. The reports will need to be run by about 3-5 people at the same time
- What OS and database type should I use (I was looking at Windows/Linux and MS SQL/MySQL)
- If we run these types of reports, can we use the same SQL server to run/store other applications (eg. CRM, etc) with no real impact on performance. There would be at least 20 people using the CRM database at the same time.
Thanks for that SG. That is some serious power needed.
Just to understand how long it will take. I have a Compaq DL380 with 2GB RAM. Can you tell me more or less how long it will take to generate a report grouping a specific field and doing a total off another field, assuming it is based on the 3 million records?