Results 1 to 4 of 4
  1. #1
    Join Date
    Aug 2004
    Posts
    4

    Arrow Big database design questions

    I'm involved in a log event system and I'm studying if we would use a database or a file backend for our purposes

    We requirements are:

    - We must access log informatition in scrollable fashion, with a day granularity. In this way, we can playback the events of a day with a Java
    GUI

    - The information volume for day is aprox. 40000 events

    - The log must be "unlimited". We are dump adcquisition data from a field
    bus RTU based on a PLC with TCP Mod Bus

    I found the following troubles:

    - The database size is going to growing up day after day

    - We don't want database performance go down owing to database size

    - How can obtain the scrollable interface we need to perform playback?

    Thanks

  2. #2
    Join Date
    Sep 2002
    Location
    UK
    Posts
    5,171
    A database: managing large quantities of data is exactly what good DBMSs do best. Your scrollable interface will query data from the database using SQL.

  3. #3
    Join Date
    Feb 2004
    Location
    In front of the computer
    Posts
    15,579
    Does the data from the RTU come back as a BLOB (implying batching), or as discrete elements (implying streaming). Both problems are easily solvable, but require radically different designs.

    A single PLC, without PLAs makes me think the elements will be discrete. The real problem is that you seem to be thinking about them as BLOBs, meaning that you need application level code to handle the aggregation in real time. That alone points toward a three tier design!

    -PatP

  4. #4
    Join Date
    Aug 2004
    Posts
    4
    Our idea seems to be:

    - Adcquire data from PLC and log it in database.

    - It's important to batch events before log becouse of possibles burst of event changes on field

    - Our implementation will be based on Java, so we need some data binding to
    recover events object from database (Castor, Hibernate...)

    - We are going to develop a playback gui tool in which the user can to navigate throught the event of a selected day.


    - The information from database must be sent to gui tool, by sockets, RMI, JMS...

    - These processes ( log and playback ) could be concurrent and more than a gui client could be connected to "Log Server"

    - The client GUI interface must be responsive. We don't want to be stuck waiting and waiting from data...

    Our problem consists on how we can model the database to store these events , and how can be it's done for high log and query performance....

    You must remember that the database must be store years of log, and we don't want performance go down while time go on

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •