Hey, Veeam Fans,
I felt it would be best to start off with a customer story from the recent record-setting Hurricane Harvey that hit Houston at the end of August 2017. I received a panicked phone call from a friend of mine who is an IT Manager for a smaller Oil and Gas company with a remote office based in Houston. This firm has been a Veeam customer for years, and I had helped them setup their backup and replication solution at the head office. During the call, my friend lets me know that they had installed some remote servers at the Houston office and that it was running their core Enterprise Resource Planning (ERP) application. Everything else was moved to the cloud except for this one workload. I figured that they would be fine because I was expecting him to ask me how to double check a Veeam replica to make sure everything would come back online in the event of power loss.
What actually happened during the call was him letting me know, that due to the recent downturn in the Oil and Gas industry, a freeze on all IT spending had been put in place and that they didn’t procure additional Veeam licenses for that site. Also, they downgraded their internet connectivity to save costs and only had a 3 MB pipe. The only backups they had were also in the Houston office, and the power and UPS in the building would only survive a few hours. He told me things should be fine for flooring because they put the servers 24 inches above the ground.
What nobody expected was that a 500 year, historical/biblical rainfall total would come with Hurricane Harvey. More than 50 inches of rain fell on their area of Houston in a little under 72 hours. The only thing that my friend could do was sit and watch in horror as his offices filled up with water. I know you are asking yourselves why do these types of circumstances still occur. I’m sure it is common knowledge to have a great IT DR plan and you should always prepare for the worst. The unfortunate reality of the situation is that when it comes to business, sometimes tough decisions are required to keep them running in a downturn. Business owners will often look to invoke a “do whatever you can to keep the lights on” strategy and hope that nothing bad happens. In this case, bad did happen. Very bad, and in fact, this customer ended up having to start over again with their ERP solution in the cloud losing all their historical data.
The better choice for this customer would have been to purchase additional licenses for Veeam Backup & Replication and integrate that into their DR strategy with replication and backup copy jobs. But, as IT professionals, sometimes our hands are tied by upper management and the people that sign the cheques. To my friend’s defense, he did warn them about the consequences and “what if” scenarios. They refused to listen, and look at what has happened. Not only did they suffer an immense loss from property and plant, they also had an immense loss of intellectual property with their ERP application going down.
But even in such cases when you’re tight on a budget, you can consider the two options below that are free from Microsoft.
Hyper-V Replica
Hyper-V Replica is an integral part of the Hyper-V role. It asynchronously replicates Hyper-V virtual machines in a primary site to replica VMs in a secondary site. Starting with Windows Server 2012 R2 we had the ability to use a feature called Extended Replication. This was extremely useful because it allowed us to have two replica copies of the VMs. Now think of the scenario above, we could have set up a second Hyper-V Host server at the Houston site, replicated the VM there as a copy, then replicated back to the head office as a final DR backup target. Even though they only had a 3 MB pipe and it would take a longer amount of time to seed the replica, it would have solved our issue. Because this customer was already using Hyper-V in the Head Office, all it would have required was some additional disk space to house the VMs.
Hyper-V Replica also allows us to have planned failover events, fail-back capabilities, and everything required to get our basic DR solution up and running.
Storage Replica
With Windows Server 2016, Microsoft introduced a new component to Failover Clustering called Storage Replica. With this, we could take our target SAN and replicate it either asynchronously or synchronously to a target. This was very useful for customers looking to achieve that Always-On approach inside of their Windows Failover Cluster. What you can do with this feature is replicate from one Cluster Shared Volume (CSV) to another. Envision a situation where a customer wanted to set up a real-time copy of their data between an iSCSI array and a different Fibre Channel array. This is completely possible with Storage Replica.
I wanted to write about this scenario because sometimes putting our heads in the sand and allowing the business to dictate what can and cannot be done is not enough. It is our responsibility as IT professionals to make the right choices for the organizations that we support. Saying “I told you so” when a customer has now lost their entire ERP solution is just plain not acceptable. If they would have spent the few thousand dollars and done the right thing using Veeam, they would have saved themselves hundreds of thousands of dollars in lost downtime.
Well, I think that is about it for our time in this blog post and I really hope you enjoyed it. As always, happy learning.
See Also
