Coffeehouse Thread

13 posts

How Windows backup backs up files

Back to Forum: Coffeehouse
  • User profile image
    IDWMaster

    Hi. I've noticed that Windows Backup checks to see IF files on the hard drive have changed since the last backup, and only backs up the files that have changed, but what about sections WITHIN the file?

    I have several large VHD files which I am backing up, and it seems that it backs up each ENTIRE VHD file each time, not just the sections that have changed. This ends up taking a ridiculous amount of time to run a backup (and greatlly slows down disc I/O throughput during the backup). Is Windows Backup designed to re-copy an ENTIRE file? Wouldn't it be more efficient if it could somehow only copy the bytes in the file that have changed?

  • User profile image
    Sven Groot

    It's a trade-off. While seeing if a file has changed is (ostensibly) as simply as looking at its last modified date, to find out which sections have changed you would need to change the entire original file and the entire modified file. Needless to say, this could take a lot of I/O and time, and is only really worth it if only a small percentage of the file has changed.

     

    The designers of Windows Backup apparently took the view that, on average, it wasn't worth the overhead, but of course you can find corner cases (like VHD files) where it would've been.

  • User profile image
    JoshRoss

    You could mount the VHD and then run backup on it. This seems a lot more complicated than setting the archive attribute, but such is life.

     

    -Josh

  • User profile image
    IDWMaster

    @JoshRoss:I have lots of VHD files which I am managing for other users, so starting up the user's VM could be an invasion of their privacy, and would be a violation of my agreement with my customers. I run a website which allows users to host virtual machines using my servers. Privacy is something that me and my company take very seriously. It just seems like there should be a better way in the Windows 7/Windows Server 2008 R2 OS to back up files more efficiently.

  • User profile image
    JoshRoss

    @IDWMaster: Would it matter if the tools were automated? How about offering an agent to run on the client's vm, that used a repository that you host?

  • User profile image
    magicalclick

    I think you need VHD backup utility like how SQL has their own backup utility? As far as OS concern, both VHD and DB are a single file. Thus, it is changed and will be backup as a file.

     

    I think it is better to back them up seperately, just like DB.

    Leaving WM on 5/2018 if no apps, no dedicated billboards where I drive, no Store name.
    Last modified
  • User profile image
    ScanIAm

    , IDWMaster wrote

    @JoshRoss:I have lots of VHD files which I am managing for other users, so starting up the user's VM could be an invasion of their privacy, and would be a violation of my agreement with my customers. I run a website which allows users to host virtual machines using my servers. Privacy is something that me and my company take very seriously. It just seems like there should be a better way in the Windows 7/Windows Server 2008 R2 OS to back up files more efficiently.

    There are drive solutions that can take immediate snapshots of currently working disks and they've been around for at least a decade.  They are expensive, though.  If you want the perfect solution, don't go with the one that comes free with the OS Smiley

  • User profile image
    figuerres

    I am thinking ....  Seems like you need a good storage system with some kind of SAN

    and a backup system that uses that.

  • User profile image
    IDWMaster

    I ended up splitting the VHDs into multiple parts using a SegmentedStream (that way, one VHD spans multiple files). Then the stream of the VHD is loaded into the virtualization software and executed.

  • User profile image
    SteveRichter

    what is a segmented stream?

  • User profile image
    JoshRoss

    This might be a stupid question, but why not just use difference disks? If the drives didn't have much churn, you could save yourself a lot of time and tape.

     

    -Josh

  • User profile image
    dahat

    @IDWMaster:Is this a client or server machine you are backing up?

     

    If a client... may I suggest Windows Home Server (2011) or Windows Small Business Server 2011 Essentials? All 3 of which backup client PCs at the cluster level vs file level... this means we are far far more efficient with regards to what all we actually copy from the PC each day.

  • User profile image
    IDWMaster

    , JoshRoss wrote

    This might be a stupid question, but why not just use difference disks? If the drives didn't have much churn, you could save yourself a lot of time and tape.

     

    -Josh

    The virtualization software I'm using doesn't support differencing disks.

    , SteveRichter wrote

    what is a segmented stream?

    A SegmentedStream is a stream which automatically creates multiple files with a specified maximum size, instead of one large file.

    For example:

    SegmentedStream sampleStream = new SegmentedStream(104857600);
    sampleStream.Write(ahugebytearray,0,ahugebytearray.Length);
    
    //104857600 is the maximum length per file. If more data than this is saved to the stream, it will automatically create multiple files.

Comments closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums, or Contact Us and let us know.