https://login.veeam.com/en/oauth?client_id=nXojRrypJ8&redirect_uri=https%3A%2F%2Fwww.veeam.com%2Fservices%2Fauthentication%2Fredirect_url&response_type=code&scope=profile&state=eyJmaW5hbFJlZGlyZWN0TG9jYXRpb24iOiJodHRwczovL3d3dy52ZWVhbS5jb20va2IxOTM0IiwiaGFzaCI6IjkzYTgxMjc4LWE1Y2QtNGM2OC1iNTk0LWI0ODAxODdmMGYzMSJ9
1-800-691-1991 | 9am - 8pm ET
EN

Hyper-V Changed Block Tracking Troubleshooting

KB ID: 1934
Product: Veeam Backup & Replication
Version: 7.x, 8.x, 9.x
Published: 2014-09-30
Last Modified: 2020-11-30
Languages: FR

Challenge

Backup and/or Replication jobs report that changed block tracking is not working properly.

This article is only relevant when Veeam Backup & Replication is used with the following Hyper-V versions. In these older versions of Hyper-V there was no native CBT function, and Veeam had to use it's own custom driver.

  • Server 2008 R2
  • Server 2012
  • Server 2012 R2
Starting with Server 2016 Hyper-V has its own CBT feature which Veeam Backup & Replication will use.

Solution

I. Latest Veeam Backup & Replication Version

Ensure Veeam Backup & Replication is running on the latest build. Latest patches will be visible on the download page for Backup & Replication (https://www.veeam.com/download-version.html). You will need to login to the Veeam Customer Portal to obtain any downloads.


II. General CBT Compatibility

The following things will cause a warning message stating “failed to flush changed block tracking” when backing up VMs on a Hyper-V cluster:

  • Ensure none of the cluster nodes are:
    • in the maintenance mode
    • have the cluster service stopped
    • are powered off
    • or otherwise not accessible

  • Ensure all nodes within the cluster can communicate with each other.

    This can be tested by pinging between the nodes and using wbemtest (see http://www.veeam.com/kb1926) to connect between nodes via WMI. For wbemtest, the two namespaces you want to ensure can be connected to are root\cimv2 and root\virtualization (root\virtualization\v2 for Windows Server 2012+).

  • Make sure the hosts are not added to Veeam Backup & Replication as standalone hosts if they belong to a cluster. This prevents changed block tracking data from being replicated to all nodes. The cluster must be added to the Veeam console.


III. For VMs residing on SMBv3 storage

  • Ensure the SMB server is added to Veeam Backup & Replication.


    User-added image<

  • Ensure the shares are visible in the Backup & Replication console.


    User-added image

    User-added image


IV. Check for Manually Created Snapshots

If Hyper-V checkpoints/snapshots are manually created/deleted (merged) on VMs, you will get a message that changed block tracking is not working. This is because checkpoint/snapshot operations create an additional file (avhd/avhdx) that has to be tracked. It is added to the list of files to track (or removed in case of deletion/merge of a snapshot/checkpoint) and the changed block tracking warning should resolve the second job run after the snapshot/checkpoint was created/deleted (merged).


V. Reseting CBT

Should the information above not resolve the issue, changed block tracking can be reset for a Hyper-V cluster by performing the following steps:

    1. Right-click on each host in the cluster, select Manage Volumes and then tick the box to "Disable changed block tracking for this host (not recommended)"


      User-added image

      User-added image
       

    2. In an elevated command prompt, on each Hyper-V host in the cluster, Run sc stop VeeamFCT,  to ensure the CBT service is stopped.
    3. On each of the Hyper-V hosts in the cluster, find, export and then delete the following registry key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VeeamFCT\Parameters\CTDB
    4. Go back to the Manage Volumes for each host and uncheck the disable changed block tracking option. 
    5. Test the job again. The job may have to be run at least twice to ensure CBT is working.

VI. Reboot Each Host in the Cluster

If the issue persists, reboot each Hyper-V host in the cluster. This will typically involve migrating VMs from one host to another and rebooting the host once it does not own any VMs. Repeat these steps for all Hyper-V hosts in the cluster.

KB ID: 1934
Product: Veeam Backup & Replication
Version: 7.x, 8.x, 9.x
Published: 2014-09-30
Last Modified: 2020-11-30
Languages: FR

Couldn't find what you were looking for?

Below you can submit an idea for a new knowledge base article.
Report a typo on this page:

Please select a spelling error or a typo on this page with your mouse and press CTRL + Enter to report this mistake to us. Thank you!

Spelling error in text

Knowledge base content request
By submitting, you agree that your personal data will be managed by Veeam in accordance with the Privacy Policy.

ty icon

Thank you!

We have received your request and our team will reach out to you shortly.

OK

error icon

Oops! Something went wrong.

Please go back try again later.