Best Practices for Microsoft Deduplication

KB ID: 2023
Product: Veeam Backup & Replication
Version: All
Published: 2015-04-22
Last Modified: 2020-08-13
Get weekly article updates
By subscribing, you are agreeing to have your personal information managed in accordance with the terms of Veeam's Privacy Notice.

Cheers for trusting us with the spot in your mailbox!

Now you’re less likely to miss what’s been brewing in our knowledge base with this weekly digest

error icon

Oops! Something went wrong.

Please try again later.

Challenge

You could experience significant slowdown in performance or failures(in rare cases) when performing Backup or Backup Copy tasks to volumes with Microsoft Windows deduplication enabled where resulted backup files are larger than 1TB in size.

Cause

The issue comes from Windows deduplication limitations, which could cause significant performance degradation while reading/appending data from/to very large files (more than 1TB). MS Requirements for Data Deduplication: https://msdn.microsoft.com/en-us/library/hh769303(v=vs.85).aspx

Solution

a) Avoid growth of backup files more than 1TB on deduplication storage, split backup jobs if possible(reducing amount of VMs per job).
b) If size of VMs does not allow splitting, try to avoid Windows deduplicated volumes usage and transfer backup files to another repository without MS Windows deduplication enabled.

More information

Best practices for MS deduplicated volumes as backup repository:

1) It is important to format the volume with large NTFS File Record Segment (FRS) (4096 bytes instead of 1024 by default) as you could face NTFS limitation errors in future. You could verify FRS size with the following command:
fsutil fsinfo ntfsinfo <volume pathname>
Command to reformat NTFS volume with larger FRS (/L):
format <volume pathname> /L

2) Use maximum NTFS allocation unit size(cluster size) of 64Kb.
Considering option mentioned in pt. 1,  two commands should be united into one:
format <volume pathname> /A:64K /L

3) Avoid growth of files more than 1TB;

4) For Backup jobs preferred backup method is Forward Incremental with Active Full backups enabled.

Introduction to Data Deduplication in Windows Server 2012: http://blogs.technet.com/b/filecab/archive/2012/05/21/introduction-to-data-deduplication-in-windows-server-2012.aspx
Plan to Deploy Data Deduplication: https://technet.microsoft.com/en-us/library/hh831700.aspx
Click here to send feedback regarding this KB, or suggest content for a new KB.
To report a typo on this page, highlight the typo with your mouse and press CTRL + Enter.

Spelling error in text

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Thank you!

Thank you!

Your feedback has been received and will be reviewed.

Oops! Something went wrong.

Please try again later.

KB Feedback/Suggestion

This form is only for KB Feedback/Suggestions, if you need help with the software open a support case

By submitting, you are agreeing to have your personal information managed in accordance with the terms of Veeam's Privacy Notice.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Verify your email to continue your product download
We've sent a verification code to:
  • Incorrect verification code. Please try again.
An email with a verification code was just sent to
Didn't receive the code? Click to resend in sec
Didn't receive the code? Click to resend
Thank you!

Thank you!

Your feedback has been received and will be reviewed.

error icon

Oops! Something went wrong.

Please try again later.