For those of us who have been in the business of backing up data for a while, we’re used to making copies of data on different types of media. I won’t date myself by mentioning reel-to-reel tapes, but the journey started with tape and then backing up data to block/file-based disk storage came along. More recently, a new kid on the block has become a popular destination for backed up data, object storage.
History of Veeam and Object Storage
Veeam started to use object storage as a target when we released Veeam Backup & Replication 9.5 Update 4 in January 2019 by introducing our use of AWS S3 compliant object storage via the Capacity Tier. A year later, we launched Veeam Backup & Replication v10 which featured support for object locking to provide data immutability for object storage. Immutability provides protection from ransomware attacks by locking the objects which prevents them from being modified or deleted by malicious software or humans. Today, Veeam Backup & Replication is protecting hundreds of petabytes of customer data using object storage.
Veeam Ready options for Technology Alliance Partners
A crucial component to the success of our object storage capabilities are our Technology Alliance Partners. Amazon, Backblaze, Cloudian, Google, Hitachi, Microsoft Azure, Wasabi, and Zadara are some of the object storage providers that Veeam has partnered with.
To help our customers choose an object storage platform which is compatible with Veeam Backup & Replication, Veeam has a qualification program, Veeam Ready. For object storage, there are two categories of qualification:
Veeam Ready Object is for TAP Partners with S3-compatible object storage solutions to test their compatibility with Veeam Backup & Replication object storage capabilities.
Veeam Ready Object Immutability is an optional test that is available for those partners who support Veeam’s object immutability features. This qualification verifies compatibility with Veeam Backup & Replication implementation of S3 Object Lock abilities on object repositories, which is a key feature for ransomware protection.
For more information regarding these Veeam Ready tests and the rest of the Veeam Ready program, please visit The Veeam Ready program for Veeam Alliance Partners.
Adding a Capacity Tier to SOBR
An object storage repository is part of a Veeam Scale-out Backup Repository. Let’s take a look at what is needed to add object storage to a Veeam Backup & Replication environment.
As you can see from the architecture slide above, you can have different repository types that make up a Scale-out Backup Repository (SOBR). We can make this process simple by following from left to right.
Adding object storage-backed Capacity Tier to your Veeam infrastructure consists of a few simple steps.
- Create one or more repositories to act as the Performance Tier.
- Create an object storage repository to act as Capacity Tier.
- Create your Scale-out Backup Repository and choose the repositories you just created as the Performance Tier & Capacity Tier extents.
Now that you have your new SOBR created, you can now point your backup jobs to this SOBR.
The steps that we just discussed are for a new SOBR. What if you already have a SOBR with existing backup jobs configured to use it? You can simply add a new Capacity Tier to your existing SOBR. When you do this, Veeam Backup & Replication will detect the existing backup file in your Performance Tier extent(s). You will then be asked if you want to copy all existing backups to the Capacity Tier or just the latest backup chains.
If you choose to copy all the existing backups, every backup file within the existing Performance Tier extent(s) will be copied to the Capacity Tier. If you choose to copy the latest backup chain, Veeam Backup & Replication will copy the active backup chain only to the Capacity Tier.
I like to refer to copying all existing backups as “historical offload” and the latest as “non-historical offload.” When offloading all the backups to the Capacity Tier, you need to make sure that your Veeam Backup & Replication infrastructure has enough resources to offload the data in the time frame that you require.
SOBR Offloading of backup data to Capacity Tier
Veeam Backup & Replication uses a “SOBR Offload” background task to copy and/or move the backup data to the Capacity Tier. The SOBR Offload background task uses “Repository Tasks” to transfer the data to the object storage. When determining the number of Repository Tasks, you should follow the sizing rule that 1 Repository Task = 1 CPU core. Each core should have 2 GB of RAM allocated to it.
In addition to the SOBR Offload job(s), the Repository Task slots are used by backup jobs, restore jobs and copy jobs. Veeam Backup & Replication allocates which job gets an available repository task when the job starts as well as while it is running. This allocation is based on a precedence hierarchy. Restore jobs are the highest priority job and the SOBR Offload has the lowest priority. So, if you have any of the other job types running during your SOBR Offload window, the SOBR Offload job(s) may not get assigned enough repository slots to finish within your preferred time frame.
Performance tuning for SOBR Offload
Another factor to consider when designing your use of a Capacity Tier is the number of concurrent S3/BLOB operations connections that Veeam Backup & Replication will use to transfer data to the Capacity Tier via the SOBR Offload jobs. Veeam’s default setting is to have a maximum number of 64 S3/BLOB operations per Repository Task slot. When using on-prem object storage, we recommend keeping the maximum number of parallel to be less than 2048 which is achieved by utilizing 32 Repository Task slots. For public cloud providers like AWS S3, Microsoft Azure BLOB and Google Cloud Storage, we recommend keeping the number of connections below 6016, which means utilizing up to 94 Repository Task slots.
In Veeam Backup & Replication v11, we added a checkbox on the Object Storage Repository wizard which allows you to limit the number of task slots that are used concurrently by the Capacity Tier.
Now you can easily limit the maximum number of S3/BLOB operations as to not overload the storage systems.
The goal is to have your SOBR Offloads complete transferring data to the Capacity Tier before the next day’s SOBR Offload jobs start. The combination of determining the correct number of Repository Task slots and calculating the correct number of S3/BLOB operations that the object storage platform can handle are key to successful SOBR Offloads, as it prevents the storage from running into an overload situation with degraded performance and high latency.
Most customers will find that these default settings will work right out of the box. If you need to fine-tune with the above settings to fit your performance needs and generate the best throughput using your object storage platform, there are a number of great resources available to help you.
Additional resources
Veeam’s official documentation site is a great source of information to help with your planning and configuration. There is also the Veeam Best Practice Documentation which are best practices from field experiences.
And don’t forget to visit your object storage provider’s website for additional information. Many of our TAP partners have “Veeam landing pages” which are chock full of guides, videos and solution briefs to help you use their object storage with Veeam. A couple of great examples are:
