In my last post, I outlined a number of new enterprise scalability features: Backup acceleration technologies, full VM restore acceleration technologies and core engine enhancements. Part of the announcement for 9.5 was about the new advanced ReFS integration, which I’ll cover in this post as it is a significant technology for Veeam. Let’s dig in!

Advanced ReFS integration

The final 9.5 feature I will cover today is arguably the most amazing feature of 9.5: Advanced ReFS integration. Resilient File System (ReFS) is a new file system of Windows Server operating systems, and the Windows Server 2016 version brings new capabilities that Veeam Availability Suite 9.5 leverages in order to provide some truly incredible benefits.

The advanced ReFS integration supports ReFS volumes on internal, direct-attached storage (DAS) and Storage Spaces — both classic and Storage Spaces Direct (S2D). The benefits (explained below) will provide significantly faster synthetic full backup creation and transformation performance, as well as reduce storage requirements and improve reliability. Even more importantly, this functionality improves Availability of backup storage by significantly reducing its load — which results in improved backup and restore performance and enables customers to do much more with virtual labs running off of backup storage. This is a big deal, so let’s explore more.

Fast cloning technology

One of the hardest parts of making a backup, and in particular a synthetic full backup, is the transformation associated with moving blocks between different backups files in the incremental chain. Version 9.5 leverages the fast cloning capability in ReFS API to create synthetic backups without moving the data blocks between files, and instead, references backup file blocks already present on the volume. This means all manipulations associated with synthetic full backups are limited to metadata updates and require no actual I/O operations on backed up data.

The video below shows how the new advanced ReFS integration works compared to synthetic full backup transformations on general-purpose storage:

Below is a screenshot of the synthetic full backup being made on a VM backup on ReFS which took only 58 seconds to process three incremental backups. Compare this to over 36 minutes for the same VM backed up to an NTFS volume!

Advanced ReFS integration coming in Veeam Availability Suite 9.5

Spaceless full backup technology

Now that synthetic full backup creation and transformation are incredibly fast and not taxing on backup storage performance, you’ll probably want to have more full backups on disk — such as periodic synthetic full backups of primary backup jobs and GFS retention of Backup Copy jobs that can keep a set number of weekly, monthly, quarterly or annual backups. I have some good news for you here too: The spaceless full backup technology brought by the advanced ReFS integration allows multiple full backups on disk to share the same data blocks. Obviously, this means a significant reduction in overall storage capacity requirements for the backup repository, providing storage efficiency rivaling that of some deduplication storage systems, but without the performance impact associated with using those (data rehydration). And what’s best, all you need is a general purpose Windows Server 2016 with a bunch of internal or direct-attached disks — same as the backup server itself uses! How cool is that?

Check out this example of a ReFS repository with one synthetic full backup made:

Advanced ReFS integration coming in Veeam Availability Suite 9.5

Data integrity streams

While better performance and reduced disk space consumption is important, they are worth nothing without reliability, which is where the last advanced ReFS integration feature comes in to play. Version 9.5 will automatically enable ReFS data integrity streams on all backup files, which are used by background ReFS data integrity scanner to detect and notify you on silent data corruption. This is critical when it comes to ensuring a reliable forever-incremental data archival model for long-term retention. You can think of it as our existing storage-level corruption guard feature on steroids because it will keep verifying your entire archive, and not just the latest restore point.

Below you can see a screenshot of the Windows Event Log event from ReFS data integrity scanner catching an uncorrectable corruption of a file on simple ReFS volume:

Data Integrity Streams

All of that is available on simple ReFS volumes — but there’s even more for those using Storage Spaces, where its proactive error-correction functionality will automatically and seamlessly fix corrupted backup file blocks using a good mirror or parity copy of the affected block from another system, thus maintaining a healthy forever-incremental archive with the required level of redundancy.

As far as requirements go, they are rather simple: The backup repository must be installed on Windows Server 2016 (and most often — such as in the case of our backup appliances — it will be the same server as the backup server). This allows the repository to leverage the ReFS API that enables integration features identified above for greater performance and capacity efficiency with large-scale data sets, while ensuring data resiliency of Veeam backup data.

Do you agree that our advanced ReFS integration completely changes the game for Veeam users? Let us know in the comments!

Additional resources:

GD Star Rating
loading...
Advanced ReFS integration coming in Veeam Availability Suite 9.5, 5.0 out of 5 based on 1 rating

View posts related to category:

All Veeam Products Top List
Technology Bloggers on Air
  • Marc K

    This is really awesome.

    GD Star Rating
    loading...
  • andyr354

    So this means ReFS has moved out of “Experimental” status for Server 2016 then? I know MS did not recommend using it to heavily yet in 2012R2.

    How does using ReFS in conjunction with storage spaces affect performance?

    GD Star Rating
    loading...
    • Rick Vanover

      Good observation, Andy. This a bit forward looking, in particular Windows Server 2016 being Generally Available and Veeam Backup & Replication 9.5 as well. At that point, it’s a support installation.

      In regards to the performance of S2D and data integrity streams; we’ve not been notified of any impact in that regard. However, we’re absent of large trials of real world usage with years of data ingested due to it being new…

      GD Star Rating
      loading...
    • Anton Gostev

      ReFS is the recommended file system for Hyper-V 2016, enough said.

      GD Star Rating
      loading...
  • James Davidson

    Will this work for Cloud Connect Backup Copy jobs if the SP is using Server 2016 with ReFS?

    GD Star Rating
    loading...
    • Anton Gostev

      Yes

      GD Star Rating
      loading...
      • Rick Vanover

        And that is awesome.

        GD Star Rating
        loading...
        • Jacob Benson

          This just made my day

          GD Star Rating
          loading...
      • starforce1

        do you have to use ReFS on the source side?

        GD Star Rating
        loading...
        • Rick Vanover

          No.

          GD Star Rating
          loading...
  • Neil Murphy

    Very exciting stuff. I’ve been using Timothy Dewin’s Restore Point Simulator for sizing repositories (http://rps.dewin.me/). Do you know if Timothy plans to update it for ReFS?

    GD Star Rating
    loading...
    • Rick Vanover

      Good question Neil. I doubt it for now, as we need some “run time” of history to collect some best practices first.

      GD Star Rating
      loading...
  • jeerd

    Do the KB1745 Deduplication Appliance Best Practices apply to ReFS? In particular the Advanced Settings -> Storage?

    GD Star Rating
    loading...
    • http://www.veeam.com/ Mike Resseler

      Hi,
      No they do not because ReFS is not deduplication. In fact, because of the way the technology works (and the API calls we do) you will want to use synthetic fulls
      Cheers
      Mike

      GD Star Rating
      loading...
  • starforce1

    can you illustrate how a 14 days retention period work? What will happend to the full backup bk1 when we get to day 15?

    GD Star Rating
    loading...
    • Rick Vanover

      Nothing specifically different would happen (based on your configuration – forward/reverse incremental chain). Just that when it does need to move data, it can do so in accelerated fashion with ReFS.

      GD Star Rating
      loading...
  • Darren Toews

    Can you combine deduplication with ReFS? Either with Microsoft’s own volume based dedupe, or with a virtual appliance like HPE’s StoreOnce VSA? Or is it a case of diminishing returns (ie: actually using up more space as in compressing an already compressed file)?

    GD Star Rating
    loading...
    • Rick Vanover

      No. There are storage efficiencies for ReFS however, it’s not deduplication per se.

      As for an appliance or VSA, it’s just going to burn CPU needlessly for low gains. May be worth trying as everyone’s data is different – but this is not a scenario we are recommending.

      GD Star Rating
      loading...
      • Darren Toews

        Good to know. Thanks Rick!

        GD Star Rating
        loading...