Multi-cloud storage optimization: An untold shared responsibility

By now, most organizations that have embraced the cloud or are considering it have moved past the naive notion that "the cloud takes care of everything and we needn’t lift a finger."

Cloud service providers (CSPs) clearly delineate their responsibilities, typically categorizing them under IaaS, PaaS, and SaaS models. IaaS places a significant onus on customers, whereas SaaS tends to require the least from them. While these technological roles are transparently laid out, there's one often overlooked or implicit responsibility: managing cloud expenses. As the old movie quote says, “the bill comes due”. Sure, costs can be reined in by setting quotas and limits. However, storage presents a unique challenge. It's like being told to empty a storage unit filled with essential documents and equipment simply because you might struggle with next month’s fee. The intricacies of cloud cost optimization intensify when third-party software utilizes cloud storage on your behalf. The assumption that they’ll optimize it for you? Think again!

In the spectrum of cloud expenses, storage costs are often the most formidable, second perhaps only to compute costs for advanced tasks like generative AI. For data protection software, the pragmatic approach is to leverage object storage over block storage to minimize costs. But as operations scale, cloud architects and IT leaders must grapple with key challenges to effectively regulate cloud object storage use and its associated costs. It's imperative to understand that while object storage is economical, frequent interactions and more consumption drive up expenses. For our purposes, it’s important to note that enterprise-scale customers anticipate data protection software to uphold deduplication, immutability, and consistency in tandem with cloud object storage.

Ease of Addition vs. Complexity of Deletion: Object storage, as opposed to block storage, comprises buckets and objects. Objects can contain data fragments used across multiple backup images, but objects cannot be changed only rewritten. Some technologies might utilize snapshot chains instead of backup images which add additional complexity around removal of objects.

Intended vs. Actual Retention of Locked Backups: CSPs offer mechanisms to lock your object storage at both the bucket and sometimes at the object level. But if the immutability isn’t effectively designed by data protection vendors, a backup intended to be locked for, say, N days might end up stored for 2N or even an alarming 7N days, multiplying the object storage consumption.

Consistency in Costs and Design: Optimal cloud storage solutions should seamlessly integrate deduplication, immutability, and consistency, regardless of the cloud type, storage class, or consumption model (whether BYO or as-a-Service). As straightforward as this may sound, it's not always a given. Technologies dependent on snapshots and snapshot chains, particularly from legacy scale-out vendors, may not be inherently compatible with object storage. Their design often leans towards block storage.

How to spot a costly cloud object storage utilization?

When evaluating data protection software that seems to offer deduplication, multi-cloud support, and immutability optimized for cloud object storage, consider these red flags:

Snapshot-chain Dependencies:

• Does the technology hinge on snapshot-chain mechanisms?

• Is there a necessity to "break" these chains to ensure optimal performance, commonly referred to as RTO/RPO?

• Is there an implicit need to deploy computational instances to eliminate redundant data or to reform the snapshot chains to discard obsolete data?

• Does it necessitate reading the object storage and subsequently rewriting it?

Object and Bucket Locking Mechanisms:

• Does the software offer object and bucket-level locking to ensure that the real immutability lock duration mirrors the intended one closely?

• Are there documented insights on how snapshot chains influence cloud object storage usage?

• Does it suggest retaining the lock for double the retention period, sidestepping the nuanced management of object-level locks?

Software Variability and Consistency:

• Does the technology necessitate distinct software deployments for block and object storage?

• Does the technology necessitate writing to block storage first to then “instantly?” tier it to Object storage?

• Is there uniform support for immutability and deduplication across major cloud service providers and across all storage classes?

• Is there a vast difference for scalability limits within a single cluster between block and object storage as a backup target?

• In the context of as-a-Service backup storage, does it advise reaching out to customer support to purge unused or outdated data?

• Is there a recommendation to allocate thrice as much cloud storage compared to block or on-premises storage?

It's paramount for organizations to be aware of these nuances that invariably influence cloud object storage costs. Beware of vendors who dismiss the value of deduplication and optimization on the pretext that “cloud object storage is economical”. Such arguments may falter when sustainability goals begin factoring in scope 2 and 3 emissions. Diesel used to be cheaper than Gasoline. Look where we are today!!

With Veritas NetBackup and Veritas Alta™ Data Protection, customers can have peace of mind. Veritas stands out, offering unparalleled deduplication, immutability, and consistency in optimizing cloud object storage.

The reasons for this superiority include:

• Veritas eliminates the need for read-write operations or temporary compute instances during retirement.

• Veritas ensures minimal overhead when combining immutability with deduplication.

• Veritas maintains a consistent technological foundation across various cloud service providers, storage classes, and consumption models, whether BYO or as-a-Service.

Intrigued? Find out more ways to optimize here.