Back on July 20, we were notified by
We have repeatedly asked Tolly who is sponsoring the report but they have declined to tell us. As of this writing, Veeam has no idea who is paying Tolly for this report (but we have our suspicions). So, in an effort to be truly fair and accurate, we proposed the following to Tolly:
- Veeam co-sponsors the tests and the report, and pays half the sponsorship fee;
- Veeam participates in determining the test criteria on an equal basis with Tolly and the other sponsor; or, alternatively, test criteria are determined by a mutually agreed-upon neutral third party;
- Both vendors’ representatives are present during the tests;
- The tests are conducted in September 2011. Veeam’s key people who can participate in the project are on vacation or on other projects until September 15; and
- The tests are conducted with Veeam Backup & Replication v6, which will be released in Q4 2011.
The above proposal was not accepted by Tolly, and we were told the process was too far along with the sponsoring vendor. They did tell us that we could sponsor our own report at a later time if we like.
My question now is how fair and accurate will this report be? According to the
Finally, we provide them [Veeam] with the opportunity to give us an official commentary on their results, the essence of which is included in any documents we might publish.
It’s interesting they say the “essence of which.” To me it means that they will listen to our feedback but will edit it into an “essence” that puts their client in a better light. Likewise, “might publish” says to me that if Veeam were to come out ahead in the test, even with cherry-picked criteria, the report will never see the light of day.
It appears that Tolly has been doing this type of “fair and accurate” testing for quite some time; I just hope no one loses millions of dollars this time based on Tolly’s findings:
Steve Duplessie from ESG has a great take on head-to-head testing and why ESG doesn’t do it:
Vendors in the backup space have also had their share of problems with Tolly, including Quest:
It’s clear to us that one of our competitors intends to use this Tolly report as part of a competitive marketing campaign against us. Veeam did NOT participate in the testing and, from our vantage point, the criteria have been designed to avoid testing typical customer scenarios as well as very features that make Veeam Backup & Replication v5 stand out as the best solution on the market. So please treat any report from Tolly as suspect.