#1 Global Leader in Data Protection & Ransomware Recovery

Increase in API Calls when Performing Backups Directly to Immutable Object Storage

KB ID: 4470
Product: Veeam Backup & Replication | 12 | 12.1
Published: 2023-07-11
Last Modified: 2024-02-13
mailbox
Get weekly article updates
By subscribing, you are agreeing to have your personal information managed in accordance with the terms of Veeam's Privacy Notice.

Cheers for trusting us with the spot in your mailbox!

Now you’re less likely to miss what’s been brewing in our knowledge base with this weekly digest

error icon

Oops! Something went wrong.

Please try again later.

Challenge

When backing up directly to Object Storage that has immutability enabled, an increase in API callsspecifically PutObjectLockRetention to the Object Storage is noticed. This behavior is observed when using either a simple Object Storage Repository or when targeting a Scale-Out Backup Repository that uses an Object Storage Repository as its Performance Tier. The spike in API calls occurs at regular intervals, typically around every 10 days.

Cause

This situation is caused by the way in which backup file immutability is maintained when using Immutable Object Storage as a primary backup destination. As explained in the How Block Generation Works section of the User Guide, Veeam Backup & Replication utilizes Block Generations to extend the immutability of groups of backup files in 10-day intervals (by default). This approach is implemented to reduce I/O operations with the storage over time.

Solution

The immutability extension API calls occurring on a 10-day cycle is behavior by design.

More information can be found here: How Block Generation Works.

More Information

How to Reduce the Number of Immutability Extension API Calls

For most customers, using the software's default settings will be practical and cost-efficient. However, for customers looking to optimize their API vs Storage costs, there are options to shift the cost load from API calls to Storage. It may be possible to reduce overall costs in some cases, but each environment is different, and data size, block size, and immutability requirements must be considered carefully.

There are two ways to reduce the overall quantity of API calls needed to maintain immutability.

However, while either of these will result in fewer API calls, storage usage will increase.

 

Option 1: Reduce the Quantity of Blocks by Increasing Block Size

To decrease the number of API calls, consider increasing the block size (Storage Optimization) of the backup files generated by jobs that perform direct backups to immutable object storage. Increasing the block size used when creating the backup files will reduce the number of blocks involved in the immutability extension process but will increase the size of the incremental restore points (on average, twice as large).

The tradeoff with this method is fewer API calls but more storage usage.

Remember that changing the block size (Storage Optimization) used by a backup job will only take effect during the next Active Full backup session. Additionally, if backup copy jobs target the immutable object storage, the block size change must be implemented in the source backup jobs first, followed by forcing an Active Full backup in both the source backup and backup copy jobs.

Option 2: Adjust the Immutability Extension Frequency

The immutability extension frequency, which affects the number of API calls, can be changed by adjusting the Block Generation interval. By increasing this interval, the occurrence of immutability extensions within a given time frame can be reduced. However, this reduction in the frequency between bursts of API calls to extend immutability duration comes at the cost of setting the maximum immutability for blocks longer than the minimum immutability setting defined in the repository settings. Resulting in fewer API calls but more storage usage.

 

Example

Consider the following before adjusting the Immutability Generation Frequency.

An Immutable Object Storage Repository has been configured to have an Immutability of 50 days. When the first data block (a full backup) is uploaded, its immutability period by default is set to 50 + 10 = 60 days. The first full backup starts its generation, which will be appended with the incremental backups. All the incremental backups within the generation (that is, within the 10-day period) will have the same immutability expiration date as the full backup. For instance, a data block offloaded on day 9 will have the same immutability expiration date as a data block offloaded on day 1. Thus, we ensure that the immutability period for all the related and dependent data blocks within a generation is never less than the immutability setting (50 days).

To maintain backup consistency, Veeam Backup & Replication can extend immutability expiration for all data blocks in all backup chains (both active and inactive) and assign those blocks to a new generation. For example, within a forward incremental backup chain, the full backup file can not be removed before its dependent incremental backup files. Therefore, the immutability period must be extended for all data blocks in a given backup chain.

If one were to alter the Immutability Generation Frequency (Days) from 10 to 25, it would cause the extensions of immutability to happen less frequently. Still, it would cause some backups to have an immutability of 75 days (50 from the repository setting and 25 from the new generation value).

I understand and still wish to change the immutability generation frequency.
(click to expand and view registry value details)

To adjust the Block Generation interval, configure the following registry value on the Veeam Backup Server:

Key Location: HKLM\SOFTWARE\Veeam\Veeam Backup and Replication
Value Name: ObjectStorageImmutabilityGenerationDays
Value Type: DWORD (32-bit) Value
Value Data Default (Dec): 10 (days)

To submit feedback regarding this article, please click this link: Send Article Feedback
To report a typo on this page, highlight the typo with your mouse and press CTRL + Enter.

Spelling error in text

This site is protected by hCaptcha and its Privacy Policy and Terms of Service apply except as noted in our Privacy Policy.
Thank you!

Thank you!

Your feedback has been received and will be reviewed.

Oops! Something went wrong.

Please try again later.

You have selected too large block!

Please try select less.

KB Feedback/Suggestion

This form is only for KB Feedback/Suggestions, if you need help with the software open a support case

By submitting, you are agreeing to have your personal information managed in accordance with the terms of Veeam's Privacy Notice.
This site is protected by hCaptcha and its Privacy Policy and Terms of Service apply except as noted in our Privacy Policy.
Verify your email to continue your product download
We've sent a verification code to:
  • Incorrect verification code. Please try again.
An email with a verification code was just sent to
Didn't receive the code? Click to resend in sec
Didn't receive the code? Click to resend
Thank you!

Thank you!

Your feedback has been received and will be reviewed.

error icon

Oops! Something went wrong.

Please try again later.