I'm involved in a log event system and I'm studying if we would use a database or a file backend for our purposes

We requirements are:

- We must access log informatition in scrollable fashion, with a day granularity. In this way, we can playback the events of a day with a Java
GUI

- The information volume for day is aprox. 40000 events

- The log must be "unlimited". We are dump adcquisition data from a field
bus RTU based on a PLC with TCP Mod Bus

I found the following troubles:

- The database size is going to growing up day after day

- We don't want database performance go down owing to database size

- How can obtain the scrollable interface we need to perform playback?

Thanks

A database: managing large quantities of data is exactly what good DBMSs do best. Your scrollable interface will query data from the database using SQL.

Does the data from the RTU come back as a BLOB (implying batching), or as discrete elements (implying streaming). Both problems are easily solvable, but require radically different designs.

A single PLC, without PLAs makes me think the elements will be discrete. The real problem is that you seem to be thinking about them as BLOBs, meaning that you need application level code to handle the aggregation in real time. That alone points toward a three tier design!

-PatP

Our idea seems to be:

- Adcquire data from PLC and log it in database.

- It's important to batch events before log becouse of possibles burst of event changes on field

- Our implementation will be based on Java, so we need some data binding to
recover events object from database (Castor, Hibernate...)

- We are going to develop a playback gui tool in which the user can to navigate throught the event of a selected day.


- The information from database must be sent to gui tool, by sockets, RMI, JMS...

- These processes ( log and playback ) could be concurrent and more than a gui client could be connected to "Log Server"

- The client GUI interface must be responsive. We don't want to be stuck waiting and waiting from data...

Our problem consists on how we can model the database to store these events , and how can be it's done for high log and query performance....

You must remember that the database must be store years of log, and we don't want performance go down while time go on