Results 1 to 8 of 8
  1. #1
    Join Date
    May 2005
    Posts
    30

    Unanswered: Resultset Compression

    Hi folks,
    Just been searching and cannot find an answer to my question. The question, admittedly, is a little vague but I’m hoping you clever folks can steer me.
    Consider a scenario where I have no control over a query or its result set. Consider that this query is fine in terms of optimization, etc. but the result set can be very large (say 40 columns with upto 30000 rows). Now, consider that every row in this result set will have values that are repeated– example, a customerName varchar(50) field.

    Finally, consider that the database that was local is now moving across the world somewhere – further from a large proportion of the clients. The query will still run fine and the result set will be produced but now I need to push all that data down the wire to a client.

    My question is: is there any compression that can be plugged in at that level – on either side of the connection to the database? If that's too easy a question consider the scenario where a dblink to an Oracle database is in use in this way:

    London Client -> NY SqlServer -> London Oracle -large resultset-> NY SqlServer -> London Client


    Now, just to save you venting your you have no idea what you’re doing and should be taken far far away from a computer fury I am aware that this is not a good situation and the only advice I need (right now!!!) is about the compression options and whether they exist?

    Thanks for any thoughts.

  2. #2
    Join Date
    Jan 2007
    Location
    UK
    Posts
    11,434
    Provided Answers: 10
    Nobody I know can absord 30K rows all at once - so why is there a need to send all that much?

    Can you given them a summary/aggregated result, or page the results, or simply filter at a lower level?
    George
    Home | Blog

  3. #3
    Join Date
    May 2005
    Posts
    30
    Thanks for taking the time to reply George. It is appreciated. However, like I said, I know a million ways to improve the process in terms of criteria gathering upfront, resultset reduction, etc.

    Just right now what I'm wondering is if compression utilities exist - in general?

  4. #4
    Join Date
    Nov 2004
    Location
    on the wrong server
    Posts
    8,835
    Provided Answers: 6
    I have been meaning to read more about this...

    .NET Remoting Overview

    ...because my current application environment uses it for new development while the old code uses web services for data access.

    I am not sure if this is going to help because I have not gotten into yet and but I no the remoted stuff seems faster than the webservices. I am not sure why.yet.
    “If one brings so much courage to this world the world has to kill them or break them, so of course it kills them. The world breaks every one and afterward many are strong at the broken places. But those that will not break it kills. It kills the very good and the very gentle and the very brave impartially. If you are none of these you can be sure it will kill you too but there will be no special hurry.” Earnest Hemingway, A Farewell To Arms.

  5. #5
    Join Date
    Nov 2004
    Location
    on the wrong server
    Posts
    8,835
    Provided Answers: 6
    george, that young scalawag, does have a point. have you heard of server side paging?
    “If one brings so much courage to this world the world has to kill them or break them, so of course it kills them. The world breaks every one and afterward many are strong at the broken places. But those that will not break it kills. It kills the very good and the very gentle and the very brave impartially. If you are none of these you can be sure it will kill you too but there will be no special hurry.” Earnest Hemingway, A Farewell To Arms.

  6. #6
    Join Date
    May 2005
    Posts
    30
    Absolutely George does have excellent point and, like I said, taking the time to reply is hugely appreciated. Just still wondering if there is a compression utility that can be plugged at that low level.

    In truth my first approach will be the .net remoting approach - put the app server close tot he database build the objects from the data there, compress them, send them to the client, uncompress...etc.. Thing is I know there are products that will help with this.

    But, of course, that introduces the extra complexity of an app server, etc. I suspect the company will first want to address the serious weaknesses in their current large resultset approach before going down that road which is a good thing.


    However, I would still like to know if the utility/etc. I'm dreaming about actually exists? Does anybody know?


    ...oh, and likelihood is the remoting than web service works better because it can exchange binary streams. And probably because it is a lot more 'closed' protocol(bad use of this word but you get my point).

  7. #7
    Join Date
    Jun 2009
    Posts
    26
    I don't know of anything off the top of my head but you may be able to use a networking device like a traffic shaper to help you out. There is a considerable investment in doing this and I'm not sure how effective it will be.

  8. #8
    Join Date
    May 2005
    Posts
    30
    Thanks for all the replies on this folks. The lack of ideas kind of tells me what I needed to know.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •