The Business Challenge
For award-winning hospitality and entertainment company, Peermont Hotels, Casinos and Resorts, all components of its business run 24 hours a day, 365 days a year. With staff operating in three shifts across its 12 properties, hyper-availability of data and systems are fundamental to the success of the business.
“We do not have the luxury of a downtime window or shutting down over the weekend for system maintenance. For us, customer service is fundamental and there are no allowances for not delivering a round-the-clock value proposition,” says Ernst Karner, Group IT Manager of Peermont Group.
Even though many other organisations have only recently started embracing the need to have hyper-available data, Peermont has been operating in this always-on environment since it opened its doors more than two decades ago.
“Historically, backing up and restoring data was not an easy proposition. We had multiple vendors claiming various features that sounded nice on paper, but the reality worked quite different. Unfortunately, no organisation can claim to not have any downtime or availability issues. The same can be said for us where drive failures were quite common in the past.”
Karner says restoring data is always used as a last resort because of the impact it could have on live systems.
“Before Veeam, we had a few incidents where recovery from our backups was not possible. We did not have the level of comfort that all our backups (stretching to more than 500 virtual machines, 275 terabytes of data, and 13 data centres) were done reliably. At the time, it was impossible to guarantee the entire company being recoverable in the event of a disaster.”
Karner says he understands customer expectations and feels that it is unforgivable in this day and age for systems to go down, irrespective of the industry a business operates in. For visitors, things like internet access (which is provided free to all customers at all Peermont properties) have become a commodity. If Peermont is unable to deliver this, people start becoming very unforgiving.
“Customers expect to have access to services at any time of day or night. As such, we had to review our capacity across all spheres of the business (network, disk storage, and compute availability). For us, there is no such thing as a Plan B. Uptime is critical and we identified Veeam as our partner of choice to address our hyper-availability requirements.”
However, unlike other businesses, Peermont requires its disaster recovery facilities to remain onsite due to its unique availability requirements. If systems go down there is an immediate financial impact on the business. However, it does supplement its on-premise business continuity strategy with a hosted component as additional fail-safe. For example, if central reservations are offline, no bookings can be made through any means. Guests will simply choose another hotel to stay at.
Similarly, if slot machines go down there is a real-time impact on revenues. Peermont can potentially lose considerable gross operating profit if downtime occurs. Not being available for two to three hours could therefore be disastrous to the business bottomline. Beyond the financial, the reputational damage could also be significant.
“Customers might forgive you once if systems are not available. But if this starts becoming an ongoing concern, they will migrate away from you to a competitor who can deliver on their expectations.”
Veeam therefore had to provide a strategic solution that could allay these concerns and provide Peermont with the peace of mind needed to deliver continuous services to customers.
The Veeam Solution
As a precursor to implementing Veeam Availability Suite, Peermont embarked on a virtualisation journey in 2005. Gradually it expanded to 70 hosts across the group moving away from physical business applications.
By improving efficiencies, Peermont can also keep costs down and run analysis on data to better understand customer needs.
“We have a centralised data warehouse that collects information from all our casinos. From supplier data and procurement systems to the requests we receive from tour operators require several Extract, Transform, and Load (ETL) processes to manage effectively.”
From an intelligent data management perspective, Peermont is focused on delivering a completely integrated ecosystem designed to give customers the best value possible. It is about using the technology, data, and intelligence from the point of arrival to the point of departure.
“We identify guests from the moment they arrive at a property and tailor their experience according to the historical data and insights we have built on them. It is about using their preferences when it comes to where they dine, what they enjoy eating, what are their favourite tables to play at, and even what shows they prefer to watch, and creating a unique environment tailored to their needs.”
Customer satisfaction is therefore an essential deliverable, but this needs to be done without adding to the cost or making it a more complex environment. Beyond this balance, Peermont also needs to meet the regulatory requirements of the South African National and Provincial Gambling Boards.
He says this requires Peermont to deliver quarterly reports that show how data is backed up at casinos and illustrate its recoverability. This sees Peermont using a team of people requesting a tape from its storage facility, restoring it to a regulated part of the server, boot it up, and verify with the Gambling Board that the recovery was successful.
As part of these compliance fundamentals, Peermont must also make daily offsite backups of the regulated part of the server. Previously, this required tape cartridges to be delivered by courier to an external facility. And when the time came to restore the data, it was a time-consuming process of recalling the tapes. With Veeam Cloud Connect, this has changed as Peermont now has an agreement in place with an external cloud service for the secure storage of its regulated data. Veeam Cloud Connect completely automates the recovery and offsite storage of this data.
“Thanks to Veeam, we have been able to take a process that would take a day at each property, requiring numerous employees, and fully automate it to take 15 minutes. I sleep a lot better at night knowing I have a system that is reliable and can recover within minutes”, says Karner.
- Automation of backup environment results in time and money savings
Using Veeam, backups that previously took Peermont a day to complete at each of its properties can now be done in under 15 minutes. And with the configuration and maintenance of its hyper-available environment centralised using Veeam from the head office, Peermont can rest assured that lack of data access that could result in significant loss of revenue, is now a thing of the past.
- Reliability and recoverability of backups
Using Veeam, Peermont ensures the integrity of its backups and guarantees being able to restore critical data in the event of a disaster. With Veeam, its backups work without fail giving them confidence knowing that risk management is taken care of.
- Scalability to cater for individual property requirements
Veeam scales according to the unique needs of each property in the Peermont Group. Whether it is a smaller casino with a limited number of virtual machines or head offices with continuously expanding data that runs into the terabytes, Veeam delivers on both ends of the spectrum.