Modern Architecture Wins Every Time: Veeam Customers Unaffected
Last week Symantec
Symantec went on a campaign to scare people from using Veeam and more importantly to scare them from upgrading to vSphere 5.1. Here’s a
This is a very bold, public claim since VMware itself states in the KB that the issue is intermittent. Symantec even used this language in their original blog post that has since been updated. Here's a screen shot of the original that is quoted below.
“During our testing with the VMware vStorage 5.1 API Symantec discovered issues* that introduce data recovery risks“
“During these weeks of close examination VMware also confirmed that any 3rd party vendor depending on the current API cannot perform consistent backups and cannot ensure a reliable recovery point”
Here’s what the KB article actually says (emphasis added):
"This article describes an issue with the VMware Virtual Disk Development Kit (VDDK) that may cause backup and restore operations to hang or fail. Third-party backup vendors that are using the VMware VDDK may encounter backup or restore issues when backing up VMware vSphere environments."
So Symantec is saying the VDDK WILL cause issues and VMware is saying it MAY cause issues.
Modern Architecture Wins Every Time
The problems in the vStorage API for Data Protection (VADP) and specifically, the VDDK that Symantec pointed out do, in fact, exist. What Symantec didn’t say is that similar issues have existed in previous VDDK versions. We at Veeam took the time to let our QC double-check earlier VDDK versions, and they were able to confirm that there are issues present in previous VDDK versions that can cause VDDK calls to hang indefinitely. We are currently working with VMware to document our findings regarding previous versions of VDDK. In the meantime, all customers who are using Symantec backup products are potentially at risk.
The Technical Explanation
The reason Veeam customers are immune to the specific issues as well as any other issues is because of our architecture. Veeam was the first vendor to support VADP on vSphere 4.0 back in the beginning of 2009. To achieve that, Veeam started working with VDDK very early on, when it was still in early beta. As is usual when working with beta software, our testers encountered a number of issues. We also recognize that deadlocks and race conditions are quite common and is something that can occur in any software. Due to that, and the fact that VDDK is a frequently updated 3rd party component that we have no control of, our development made the decision to design VDDK interaction architecture in a way that allows us to easily handle unexpected VDDK issues. Our development team wasn’t able to fix these issues but they did realize that separating the VDDK calls into a separate process monitored by a watchdog process will help to dramatically increase overall reliability. In case this process hangs, we simply kill the process without affecting the job. With an alternative architecture not featuring such isolation, VDDK code hang will cause the whole backup and restore “job” to hang just as the VMware KB article explains.
Additionally, Veeam sets extra timeouts on some critical VDDK calls to avoid possible deadlocks. In short, Veeam customers are immune to VDDK issues while Symantec customers need to go back and verify the recoverability all of their VMware VM backups, as per Symantec’s own tweet.
Because Veeam is Built for Virtualization™ and not simply a retrofit to an outdated backup architecture our customers can sleep easy knowing that Veeam is #1 for virtualization backups. Since its release, Veeam Backup & Replication 6.5 has been downloaded more than 50,000 times, and we estimate that at least half are evaluating or using this version against vSphere 5.1 without any issues whatsoever. The numbers speak for themselves!