Coffeehouse Thread

10 posts

Forum Read Only

This forum has been made read only by the site admins. No new threads or comments can be added.

Remote Differential Compression

Back to Forum: Coffeehouse
  • User profile image
    Rossj

    This looks like it will be introduced in Longhorn Server and looks kinda cool* why I hadn't heard about it before I don't know.  Don't you think it is funny how everyone's killer feature is different in both functionality and scale?

    Anyone else got a use for this (and want to help me 'persuade' the guys that a port to 2003 would be cool)? Persuade isn't a synonym for pester - honest.

    Jason wrote:

    What is Remote Differential Compression? Remote Differential Compression (RDC) allows applications to synchronize data between two computers in an efficient manner. The synchronization efficiency is made possible by using compression techniques to minimize the amount of data sent across the network.

    What makes RDC different from other differencing mechanisms? RDC is different from patching-oriented differencing mechanisms, such as Binary Delta Compression (BDC), which are designed to operate only on known versions of a single file. RDC makes no assumptions about file similarity or versioning. Because differences between files are computed on the fly, RDC is ideally suited for synchronizing files that are different or have been updated independently.


    * I've already emailed Jason asking if there will be a backport because I really could do with this soon (end of week actually) - to the point where I'd actually delay the product for it (if it works Wink )

  • User profile image
    jolson88

    Haha. I don't know why it makes me smile to see a Niner quote my blog Tongue Out. Remote Differential Compression is actually one of the specific technologies that I'm an evangelist for.

    It will be back ported to Windows Server 2003. The caveat is that it will be in Windows Server 2003 R2 (not pre-R2). Also, it will be available on both Vista and Longhorn Server. So, I'm not sure if that is applicable for you or not.

    When "synch"-ing large files between two places, RDC is a HUGE boost in performance (some of the MSDN articles talk about why, how, etc.).

    One of the interesting usages of RDC that I'm waiting to see someone exploit is the fact that RDC doesn't technically have to use "files." Since the COM server you implement actually does the file reading/writing and such, I'd imagine you could use it to sync things that aren't files. I'll leave the rest to your imagination Smiley.

    * Weird, what method did you use to email me? I haven't gotten it yet Perplexed 

  • User profile image
    Rossj

    JasonOlson wrote:
    

    Haha. I don't know why it makes me smile to see a Niner quote my blog . Remote Differential Compression is actually one of the specific technologies that I'm an evangelist for.

    It's quite an interesting blog, I just subscribed, although I won't comment because I have more than enough internet accounts and don't need any more Sad

    JasonOlson wrote:
    

    It will be back ported to Windows Server 2003. The caveat is that it will be in Windows Server 2003 R2 (not pre-R2). Also, it will be available on both Vista and Longhorn Server. So, I'm not sure if that is applicable for you or not.

    When "synch"-ing large files between two places, RDC is a HUGE boost in performance (some of the MSDN articles talk about why, how, etc.).

    Hmm, I can *probably* cope with that,  the fact I am moving around lots of 100Mb movie files is what piqued my interest in the first place Smiley

    JasonOlson wrote:
    One of the interesting usages of RDC that I'm waiting to see someone exploit is the fact that RDC doesn't technically have to use "files." Since the COM server you implement actually does the file reading/writing and such, I'd imagine you could use it to sync things that aren't files. I'll leave the rest to your imagination .

    That's quite a nice model, and the sort of thing I'd love to see more of from Microsoft, I guarantee *someone* will implement the basics fairly early on but the flexibility is there (without any crazy configuration or complexity) to do what you want with it.

    Do you have any performance stats for it (on a typical LAN - if any such thing exists)? I'm wondering how 'real-time' you could get it (CPU and network permitting) ...

    JasonOlson wrote:
    

    * Weird, what method did you use to email me? I haven't gotten it yet 



    The C9 mail icon next to your blog icon, I think it came from my gmail address and I've had trouble mailing into Microsoft with that in the past. I'll change my C9 mail to the @mac.com one Smiley

  • User profile image
    jolson88

    Rossj wrote:
    

    It's quite an interesting blog, I just subscribed, although I won't comment because I have more than enough internet accounts and don't need any more



    Yeah, that's one of my annoyances with the MSDN blogs. If you want the "full package" (including all the personal ramblings and the like), you can go subscribe to the original source: http://www.managed-world.com. Everything on the MSDN blog is simply cross-posted from there anyways.

    Rossj wrote:
    

    Hmm, I can *probably* cope with that,  the fact I am moving around lots of 100Mb movie files is what piqued my interest in the first place



    Just be aware, this may not be want you want then. It's almost unfortunate that they used the term "Compression" because it's less of a compression algorithm, and more of a synchronization algorithm. RDC really shines when you have a source file on a server, and a destination file on a local computer you want to sychronize. In this situation, RDC basically just copies down the individual bytes that are different between the two files.

    If you have a file that's on the server, but not on the client (or vice versa), then you will have to copy the entire file down.

    Rossj wrote:
    

    Do you have any performance stats for it (on a typical LAN - if any such thing exists)? I'm wondering how 'real-time' you could get it (CPU and network permitting) ...



    Here are some numbers from a study done on distributed file synchronization:

    - Over a DSL connection, saving all changes for a 3.5MB PPT file took 70 seconds. Saving just the changes via RDC took less than a second.

    And some reduction factor information (RDC vs. Full File Transfer):

    .VSD [318k] - 3x
    .DOC [489k] - 13x
    .DOC [2.5M] - 15x
    .MPP [241k] - 17x
    .PPT [594k] - 31x
    .XLS [2.4M] - 30x
    .ZIP [348k] - 41x
    .HTM [425k] - 92x
    .PPT [3.9M] - 292x
    .PST [293M] - 409x

    Now, I'm not exactly sure the details of the study that was done to generate these numbers. I'm fairly sure though that it was "synchronizing" a file that already exists in two places. I'm also not sure what the number of changes were, so keep that in mind when looking at those numbers Smiley.

  • User profile image
    Rossj

    JasonOlson wrote:
    

    Just be aware, this may not be want you want then. It's almost unfortunate that they used the term "Compression" because it's less of a compression algorithm, and more of a synchronization algorithm.


    Oh yeah I totally got that, I was thinking RSync on Steroids Smiley We have a lot of large files that we distribute, but then at a later stage we want to distribute changes to those files - typically changes to the metadata or modified copies of those files.  It is for digital signage where we have to get video to a machine and then update that video if it has changed (which happens - A LOT) on all 300-odd machines.  You can probably see why I am so interested now Smiley


  • User profile image
    blowdart

    What is the best way to update an existing 2003 server to RC2?

  • User profile image
    AndyC

    blowdart wrote:
    What is the best way to update an existing 2003 server to RC2?


    1) Buy licence
    2) Pop CD in
    3) ?
    4) Success!

    Smiley

    It's a pretty painless upgrade process to be honest.

  • User profile image
    blowdart

    AndyC wrote:
    
    blowdart wrote:What is the best way to update an existing 2003 server to RC2?


    1) Buy licence
    2) Pop CD in
    3) ?
    4) Success!



    It's a pretty painless upgrade process to be honest.


    So slap it over the existing setup? Cool. Time to break the mail server at home.

  • User profile image
    AndyC

    blowdart wrote:
    

    So slap it over the existing setup? Cool. Time to break the mail server at home.


    The R2 setup comes on two CDs; the first installs plain vanilla Server 2003 SP1, the second installs the R2 bits; so popping the second CD in to just add the new bits should work, IIRC.

  • User profile image
    W3bbo

    AndyC wrote:
    
    blowdart wrote: 

    So slap it over the existing setup? Cool. Time to break the mail server at home.


    The R2 setup comes on two CDs; the first installs plain vanilla Server 2003 SP1, the second installs the R2 bits; so popping the second CD in to just add the new bits should work, IIRC.


    That's how TabletPC and Media Center installation works too, but the second CD is only accepted during setup.

    ...whatever Smiley

    Is there an R2 "upgrade edition" at all? Seems a bit of a waste to shell out £800 just for POSIX support.

Conversation locked

This conversation has been locked by the site admins. No new comments can be made.