Page 1 of 2 12 LastLast
Results 1 to 15 of 29
  1. #1
    Join Date
    Nov 2007
    Posts
    10

    OLTP-OLAP question.Please Help!

    I would much appreciate it if someone answered a question i have concerning OLTP systems and OLAP/Data warehouses.On which of the two systems(OLTP or OLAP or both) can we implement the 5-minute rule and why?The 5-minute rule says 'Data referenced every five minutes should be memory resident' for those of you who don't remember.Thanks for your help in advance.

  2. #2
    Join Date
    Apr 2002
    Location
    Toronto, Canada
    Posts
    20,002
    you can implement that rule on OLTP systems

    OLTP = on line toast processing

    when the query lands with the buttered side down, you have 5 minutes to attempt to re-run it before the cooties take hold
    rudy.ca | @rudydotca
    Buy my SitePoint book: Simply SQL

  3. #3
    Join Date
    Jun 2007
    Location
    London
    Posts
    2,527
    Tasidious

    I suspect whoever invented this 5 minute rule had no idea how most modern databases work.

    When you first grab some data from the database, this data is moved from the disks to the cache (memory). If you then access this data again then the server will see that the data is in the cache and pull it directly from there rather than going to the disks. This means your most frequently accessed data (most likely the data you accessed in the last 5 minutes) will always be available in memory anyway. Most servers will store around 2gb of cache so that's a fair amount of data but YMMV. Data will only disappear from the cache if you (or others) start accessing lots of data from other tables more frequently.

    You can tell MySQL (and perhaps others) to store tables directly in memory rather than on disk but remember that even memory and disk space can sometimes blur together when considering swap space etc. I'd imagine it would be quite difficult writing a system to hold such data in memory explicitly and seeing as it takes no effort at all to use the standard caching methods available on any database I'd suggest you use that. Most systems allow you to alter the size of the cache if you really need to hold huge amounts of data in memory (assuming the memory is available on the server). Most databases do a great job in managing their cache and will always do a better job than any simple "5 minute rule".

    As a general rule I'd suggest you cut your toast up into "fingers" then you may find each slice easier to hold and less likely to drop, it also reduces the chance of catastrophic toast loss. As a side benifit you'll also find that the slimmer "fingers" also allow you to more easily dunk the toast into the yokes of boiled eggs.

    Mike
    Last edited by mike_bike_kite; 11-11-07 at 09:43.

  4. #4
    Join Date
    Nov 2007
    Posts
    10
    Quote Originally Posted by mike_bike_kite

    I suspect whoever invented this 5 minute rule had no idea how most modern databases work.

    Well in fact it's an assignment i have.I have to read a paper about the 5-minute rule and i have to answer the above question.So it's implemented on OLTP systems but a more detailed reason why would be much appreciated.What about OLAP systems?I guess the 5-minute rule is not implemented on OLAP?Why is that? I searched google a lot before posting here.I couldnt find a thing.

  5. #5
    Join Date
    Apr 2002
    Location
    Toronto, Canada
    Posts
    20,002
    what does your paper say about it?
    rudy.ca | @rudydotca
    Buy my SitePoint book: Simply SQL

  6. #6
    Join Date
    Nov 2007
    Posts
    10
    The paper is about the 5 minute rule which you can find here http://research.microsoft.com/~gray/...MinuteRule.doc
    Using the 5 minute rule we must answer the following question.
    On which of the following systems is the 5 minute rule implemented:
    a.OLTP systems
    b.OLAP/Data warehouses
    c.Both
    d.None of the above.
    Justify your answer.

  7. #7
    Join Date
    Sep 2003
    Location
    The extremely Royal borough of Kensington, London
    Posts
    778
    Quote Originally Posted by From the article you made reference to
    A Tandem disc, and half a controller, comfortably deliver 15 accesses per second and are priced at 15K$ for a small disc and 20K$ for a large disc (180Mb and 540Mb respectively)
    When, I wonder, was the article written?

    Your question about which database environment would the so called "Five minute rule" apply is logically impossible to answer, for the reason that the variable for which the answer depends, cannot be defined from the information you have provided, and indeed in real life, can only be estimated at best. The variable relates to what I shall call the "scope of data access" within a system. This can be interpreted in many way, though for simplicity I shall define as referring to the breadth of data access that a system handles.

    This concept is critical to all forms of caching or in the management of data storage between physical memory and disk. In determining a strategy for the management of data within memory, one must also consider in conjunction to the scope of data access, the amount of physical memory available, another variable that the article conveniently omitted.

    I shall leave you with the following situation, to which I will be interested in your answer.

    I have 10 units of memory available to my data management system. Of all the operations processed by this system, two read queries, A and B, are executed 500 times each per second. We can therefore classify that these are indeed candidates for permanent residence in memory, as per the five minute rule mentioned in your referenced document.

    After an analysis of the memory consumption of these queries, it is revealed that each result set occupies eight units of memory. That is, eight units of memory are required to store the result set of either query, A or B, in physical memory. Intuitively it is not possible to store the result sets of both queries in physical memory at the same time, for 2 * 8 = 16 units of memory, which is greater than the total of 10 units available.

    Therefore my question to you is, which of these queries should be stored in main memory?

    In real life, companies that make the big profits ($Billions+) employ an advanced technique in the field of information management, which is to allocate resources, primarily in the form of individuals, to understand the data, the processes that act on that data, and the strategic objectives of the company in managing the data. With this analysis of information, decisions can then be made about how to most effectively manage the information. One example may involve deciding which subset of data should be stored in physical RAM during the hours of 9am and 5pm.
    Last edited by r123456; 11-11-07 at 13:05.
    Bessie Braddock: Winston, you are drunk!
    Churchill: And Madam, you are ugly. And tomorrow, I'll be sober, and you will still be ugly.

  8. #8
    Join Date
    Sep 2003
    Location
    The extremely Royal borough of Kensington, London
    Posts
    778
    I guess the 5-minute rule is not implemented on OLAP?Why is that?
    Why, for what reason known to man, should it be implemented? Because the author of some random document on the internet naively believes that a "five minute rule" is an effective strategy in the management of data?

    Really, when you ask a question such as to understand the reasons why a successful global corporation may not implement an idea that just happens to be published to the internet (there is no restriction on what can be published to the internet, think mySpace for an example of a collection of utter rubbish), as if to suggest that the internet is the authoritative source of truth, how can you expect people to take the time to deliver a serious reply.

    And just for completeness, if companies did behave as you suggest, we would still be living in the stone age. There is a very important rule to remember about written material, especially when it is available on the internet, which is:

    Just because something is written down does not confirm that the expressed ideas, arguments or assertions are true or of value.

    Capitalism succeeds for many a reason, and adherence by members of successful societies to the above statement is one of them.

    If you ask a stupid question, you will almost invariably receive an answer given in jest.
    Last edited by r123456; 11-11-07 at 13:12.
    Bessie Braddock: Winston, you are drunk!
    Churchill: And Madam, you are ugly. And tomorrow, I'll be sober, and you will still be ugly.

  9. #9
    Join Date
    Nov 2007
    Posts
    10
    Quote Originally Posted by r123456
    Why, for what reason known to man, should it be implemented? Because the author of some random document on the internet naively believes that a "five minute rule" is an effective strategy in the management of data?

    Really, when you ask a question such as to understand why do successful global corporations not immediately implement an idea because it is published to the internet (there is no restriction on what can be published to the internet, think mySpace for an example of a collection of utter rubbish), how can you expect people to take the time to deliver a serious reply.

    And just for completeness, if companies did behave as you suggested in your original post, we would still be in the stone age. There is a very important rule to remember about written material, especially when it is available on the internet, which is:

    Just because something is written down does [B] not [b] confirm that the expressed ideas, arguments or assertions are true or of value.

    Capitalism succeeds for many a reason, and adherence by members of successful societies to the above statement is one of them.

    If you ask a stupid question, you will almost invariably receive an answer given in jest.
    Dis you see my last post?It's for an assignment i have!Dont rush into conclusions!Blame the teacher who gave me the assignment....
    Last edited by Tasidious; 11-11-07 at 13:08.

  10. #10
    Join Date
    Sep 2003
    Location
    The extremely Royal borough of Kensington, London
    Posts
    778
    I did indeed, place the blame on the teacher. I feel sorry for you that you have to try to answer such questions that are logically impossible. Perhaps you should raise this point with the teacher?
    Bessie Braddock: Winston, you are drunk!
    Churchill: And Madam, you are ugly. And tomorrow, I'll be sober, and you will still be ugly.

  11. #11
    Join Date
    Jun 2007
    Location
    London
    Posts
    2,527
    On which of the following systems is the 5 minute rule implemented:
    a.OLTP systems
    b.OLAP/Data warehouses
    c.Both
    d.None of the above.
    Justify your answer.
    d) None of the above

    Reasons:
    • The rule belongs to a different age - somewhere between the age of cuneiform and steam.
    • The Rule is impossible to enforce ie what would you do if you read more data in 5 min than can be stored in memory?
    • The huge effort required to code this 5 minute rule.
    • The pointless of doing so as caching makes this 5 minute rule obsolete (it sounds like you may need to explain caching to your teacher).
    • Also the distinction between memory and disk space is a little blurred on servers where swapping tends to be used to artificially extend the memory.
    I can't guarantee an A+ but at least the teacher may be a little more careful and give you better questions that are more suited to the the modern world.

    When, I wonder, was the article written?
    I think it said 1985 which, in dog years, is a hell of a long time ago.

    Mike

  12. #12
    Join Date
    Jan 2007
    Location
    UK
    Posts
    11,434
    Quote Originally Posted by mike_nike_kite
    The rule belongs to a different age - somewhere between the age of cuneiform and steam.
    Lol!

    This message is too short
    George
    Home | Blog

  13. #13
    Join Date
    Nov 2007
    Posts
    10
    Thanks for your answer Mike.Things are becoming more clear now.If someone else has a different opinion on this matter feel free to post it.

  14. #14
    Join Date
    Nov 2007
    Posts
    10
    Quote Originally Posted by r937
    you can implement that rule on OLTP systems

    OLTP = on line toast processing

    when the query lands with the buttered side down, you have 5 minutes to attempt to re-run it before the cooties take hold
    According to Mike you are wrong...At first i thought that the answer a is correct but after Mike's post i have my doubts.
    OLTP systems are update-oriented, with constant additions,updates and deletions to their records whereas OLAP are updated infrequently using large load operations.So i was going to go for answer a but Mike's post made me think about it all over again...

  15. #15
    Join Date
    Jun 2007
    Location
    London
    Posts
    2,527
    Originally posted by Tasidious
    According to Mike you are wrong
    I think r937 was actually joking. There are lots of things Rudy and I may differ on but I doubt this is one of them. I also have to be very very nice to him else he'll ban me

    Mike

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •