For award-winning hospitality and entertainment company, Peermont Hotels, Casinos and Resorts, all components of its business run 24 hours a day, 365 days a year. With staff operating in three shifts across its 12 properties, hyper-availability of data and systems are fundamental to the success of the business.
“We do not have the luxury of a downtime window or shutting down over the weekend for system maintenance. For us, customer service is fundamental and there are no allowances for not delivering a round-the-clock value proposition,” says Ernst Karner, Group IT Manager of Peermont Group.
Even though many other organisations have only recently started embracing the need to have hyper-available data, Peermont has been operating in this always-on environment since it opened its doors more than two decades ago.
“Historically, backing up and restoring data was not an easy proposition. We had multiple vendors claiming various features that sounded nice on paper, but the reality worked quite different. Unfortunately, no organisation can claim to not have any downtime or availability issues. The same can be said for us where drive failures were quite common in the past.”
Karner says restoring data is always used as a last resort because of the impact it could have on live systems.
“Before Veeam, we had a few incidents where recovery from our backups was not possible. We did not have the level of comfort that all our backups (stretching to more than 500 virtual machines, 275 terabytes of data, and 13 data centres) were done reliably. At the time, it was impossible to guarantee the entire company being recoverable in the event of a disaster.”
Karner says he understands customer expectations and feels that it is unforgivable in this day and age for systems to go down, irrespective of the industry a business operates in. For visitors, things like internet access (which is provided free to all customers at all Peermont properties) have become a commodity. If Peermont is unable to deliver this, people start becoming very unforgiving.
“Customers expect to have access to services at any time of day or night. As such, we had to review our capacity across all spheres of the business (network, disk storage, and compute availability). For us, there is no such thing as a Plan B. Uptime is critical and we identified Veeam as our partner of choice to address our hyper-availability requirements.”
However, unlike other businesses, Peermont requires its disaster recovery facilities to remain onsite due to its unique availability requirements. If systems go down there is an immediate financial impact on the business. However, it does supplement its on-premise business continuity strategy with a hosted component as additional fail-safe. For example, if central reservations are offline, no bookings can be made through any means. Guests will simply choose another hotel to stay at.
Similarly, if slot machines go down there is a real-time impact on revenues. Peermont can potentially lose considerable gross operating profit if downtime occurs. Not being available for two to three hours could therefore be disastrous to the business bottomline. Beyond the financial, the reputational damage could also be significant.
“Customers might forgive you once if systems are not available. But if this starts becoming an ongoing concern, they will migrate away from you to a competitor who can deliver on their expectations.”
Veeam therefore had to provide a strategic solution that could allay these concerns and provide Peermont with the peace of mind needed to deliver continuous services to customers.
As a precursor to implementing Veeam Availability Suite, Peermont embarked on a virtualisation journey in 2005. Gradually it expanded to 70 hosts across the group moving away from physical business applications.
By improving efficiencies, Peermont can also keep costs down and run analysis on data to better understand customer needs.
“We have a centralised data warehouse that collects information from all our casinos. From supplier data and procurement systems to the requests we receive from tour operators require several Extract, Transform, and Load (ETL) processes to manage effectively.”
From an intelligent data management perspective, Peermont is focused on delivering a completely integrated ecosystem designed to give customers the best value possible. It is about using the technology, data, and intelligence from the point of arrival to the point of departure.
“We identify guests from the moment they arrive at a property and tailor their experience according to the historical data and insights we have built on them. It is about using their preferences when it comes to where they dine, what they enjoy eating, what are their favourite tables to play at, and even what shows they prefer to watch, and creating a unique environment tailored to their needs.”
Customer satisfaction is therefore an essential deliverable, but this needs to be done without adding to the cost or making it a more complex environment. Beyond this balance, Peermont also needs to meet the regulatory requirements of the South African National and Provincial Gambling Boards.
He says this requires Peermont to deliver quarterly reports that show how data is backed up at casinos and illustrate its recoverability. This sees Peermont using a team of people requesting a tape from its storage facility, restoring it to a regulated part of the server, boot it up, and verify with the Gambling Board that the recovery was successful.
As part of these compliance fundamentals, Peermont must also make daily offsite backups of the regulated part of the server. Previously, this required tape cartridges to be delivered by courier to an external facility. And when the time came to restore the data, it was a time-consuming process of recalling the tapes. With Veeam Cloud Connect, this has changed as Peermont now has an agreement in place with an external cloud service for the secure storage of its regulated data. Veeam Cloud Connect completely automates the recovery and offsite storage of this data.
“Thanks to Veeam, we have been able to take a process that would take a day at each property, requiring numerous employees, and fully automate it to take 15 minutes. I sleep a lot better at night knowing I have a system that is reliable and can recover within minutes”, says Karner.
Peermont Hotels, Casinos and Resorts is an award-winning hospitality and entertainment company which operates 12 properties located across South Africa, Botswana, and Malawi. Renowned for its excellence in design, development, management, ownership, and operation of multifaceted hospitality and gaming facilities, guests partake in fine dining, relaxing hotel stays, exciting casino action, live entertainment, soothing spa treatments, efficient conferencing or sporting activities — offered in unique safe and secure, themed settings.
The competitive status of Peermont is driven by a strong, entrepreneurial management team and capable staffing driven by a desire to succeed. As part of this focus on delivering exceptional service and a guest-centric approach, Peermont needs to comply with legislation from the South African National and Provincial Gambling Boards. This requires the backup and availability of gaming system data. To this end, all required backups need to be tested on an ongoing basis to ensure recoverability of information.
Adding further pressure to the environment is a constant expectation from guests that the business will not experience any downtime. Whether it is complimentary WiFi inside rooms or slot machines, restaurant payments or entertainment bookings, Peermont must always deliver an always on experience for guests.