This my problem: I need to track an object which emits 1 million different variables several times a day (in average 10 times a day).
Technically, it is possible to split the variable into groups, and then put each group in a table. but I can't imagine myself create the schema with one million columns...
Are relational databases the best choice to deal with that?
I thought I can create a table like a dictionary with two columns <key, value> and though have 1 million rows instead of 1 million columns... But it will be difficult to write queries...
Last edited by poplop; 02-16-10 at 06:51.
Reason: message was trunked
It is a kind of robot explorer and the variables describe its state and the environment.
No matter what are the variables, and for sure, it won't be 1 million variables but maybe 990000 or something like that...
I wonder if the present DB software will be useful for that
You aren't really looking at a typical relational solution.
The sort of solution for this sort of problem are traditionally either EAV or an XML column. You will need to research these terms. EAV is just an extension of your two column <key value>. Note that both of these solutions come with health warnings, but with a problem like this you have to compromise somewhere.
I still don't believe there's a million variables. Can you just list a few of the variables (not all of them) so we can get an idea of what data you're holding. Is it the environment around the robot? - in which case it should just be the time, x pos and y pos, the type of sensor and the value read. If it's internal sensors on the robot then it's just time, sensor id, reading. If it's camera or ultrasonic data then shouldn't you be analysing the data before storing any info.
What environment does the robot explore? what type of sensors are you using? What will you do with the data? Is it a real robot (photo please) or just a pretend one?