Results 1 to 7 of 7
  1. #1
    Join Date
    Oct 2003
    Posts
    268

    Unanswered: Basic logging/processing db question

    I have many worker systems logging data to a central database.
    The central database needs to periodically process all that data. During this processing time, new records can't be added part way through and the workers still need to log the data somewhere.

    I was thinking of the following approach:

    1) Rename the table the worker logs to. Let's say from LogTable to ProcessingTable
    2) Immediately create a new and empty table for the worker to log to (LogTable). Possibly wrap this and the preceeding rename step in a transaction which may provide some atomicity.
    3) Process the ProcessingTable
    4) Drop ProcessingTable.

    To the db experts here, does this sound like a decent approach or are there problems with this?

  2. #2
    Join Date
    Feb 2004
    Location
    San Antonio, TX
    Posts
    565
    Sounds a bit dirty
    can you give a bit more info on the type of processing. is this cube population or just calculations for oltp?
    how often? and how much(data)?

    I would prefer to put the new data in table A
    use dts to process the data every hour and place it in the destination table b
    and have querying happen full time on table b

    this way you only have 1 hour of latency in your data
    if the info is not as volitile then schedule processing more closely together.

  3. #3
    Join Date
    Jul 2003
    Location
    San Antonio, TX
    Posts
    3,662
    Problem #1 - you haven't paid the bill. Do you need an official invoice?
    Problem #2 - you can't count on free advice while supporting your production.
    Problem #3 - when advice is given and it's proven to be the right solution, - you need to follow it rather than getting yourself into unchartered waters with bigger problems on your hands.

    Is that enough?
    "The data in a record depends on the Key to the record, the Whole Key, and
    nothing but the Key, so help me Codd."

  4. #4
    Join Date
    Feb 2004
    Location
    San Antonio, TX
    Posts
    565
    sounds to me like someone is preaching the brainwashing he just got at his "Fanatical Customer Service" Based Job

  5. #5
    Join Date
    Jul 2003
    Location
    San Antonio, TX
    Posts
    3,662
    No, Scott, this has nothing to do with "Fanatical" brainwashing, it has to do rather with the guy paying the bill for the services that he received from me. Rings the bell now?
    "The data in a record depends on the Key to the record, the Whole Key, and
    nothing but the Key, so help me Codd."

  6. #6
    Join Date
    Feb 2004
    Location
    San Antonio, TX
    Posts
    565
    oh what was i thinking poking fun at you?

    i'm just full of sour grapes because they ...... well ill save that story for next week when i get back and we get drunk.

  7. #7
    Join Date
    Jul 2003
    Location
    San Antonio, TX
    Posts
    3,662
    Hey, can't wait!!!

    Kurt,

    It's really not difficult, just dial the number and say that you guys were VERY busy, and the hole thing will go away, you know?!
    "The data in a record depends on the Key to the record, the Whole Key, and
    nothing but the Key, so help me Codd."

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •