Storage economics

Backup retention costs: What another year really adds

Extending retention is not linear in time. How to work out what another year of backup retention actually costs.

7 min read
A long receding row of glowing teal data blocks fading into darkness along a dark server aisle

A request to extend backup retention from one year to two is usually costed by doubling something. The instinct is understandable, since retention is expressed in time and time is linear. Storage consumption is not. What a twelfth extra month adds depends on what is actually sitting in the retained set at that depth, how well it still deduplicates against everything around it, and whether the copies are locked against deletion.

The question is answerable, but not from the retention setting alone. Two environments with identical protected data and identical retention can differ substantially in what the extra year costs, because the retained set is composed differently. At short depths it is mostly incremental data referencing recent fulls. At long depths it is mostly fulls kept as weekly, monthly and yearly restore points, and those behave nothing like increments.

There is also a second bill, paid in operations rather than capacity. A longer tail enlarges the search space during a restore, lengthens the next migration, and, where copies are immutable, turns a storage decision into a commitment that cannot be reversed until the lock expires.

Why the extra year is not proportional

Retention costs are driven by restore point count and composition, not by elapsed time. A daily schedule retained for a month produces many closely related restore points. A grandfather, father, son schedule retained for years produces far fewer, and far less related ones. Adding twelve months at the short end adds daily points; adding twelve months at the long end adds monthly and yearly fulls.

Those two additions differ in cost per restore point by a wide margin. An incremental point stores the blocks that changed since its predecessor. A full point represents the whole protected set at a moment in time, and deduplicates against its neighbors only to the extent the underlying data has not changed. The further apart two fulls sit, the less they share.

The correct question is therefore not how long to retain but which restore points the extra period contains. How restore points accumulate under a schedule is the subject of retention and recovery points, and that composition is the first input to any retention cost estimate.

What sits in the extra year

Start by listing the restore points the policy will hold at the new depth, by type. For most schedules that is a short list: some weekly fulls, some monthly fulls, and one or two yearly fulls. Daily points have usually aged out long before the second year, so the extra year is almost entirely fulls.

Then establish how those fulls are produced, since the mechanism determines what is stored. Chain structure on object storage differs from that on a file repository, and block generation affects how long blocks stay referenced. The behavior of backup chains on object storage is worth confirming for the version in use, since implementations vary and the capacity consequence follows from the implementation.

The output is a count of retained fulls added by the extension and a statement of how each is stored. That is a concrete quantity, unlike a retention period, and it can be multiplied by a measured size rather than an assumed one.

How deduplication behaves over a longer window

Deduplication and compression effectiveness are not constant across a retention window. Blocks common to restore points taken days apart are plentiful. Blocks common to points taken a year apart are fewer, since files have been rewritten, applications upgraded and databases reorganized. A ratio observed over a month is an optimistic estimate of the ratio that applies over a year.

Encryption and source side compression compound this. Data that arrives already encrypted or compressed reduces poorly regardless of window length. An extension priced on a ratio measured over a short window will understate what the extra year consumes.

The measurement that answers this needs no new tooling. Compare the stored size of same type restore points taken at increasing separations, and the shape of the curve is the answer. It will not be linear, and knowing its shape is worth more than any published ratio.

What changes at longer depthWhy cost is not proportionalWhat to measure in the environment
Mix of fulls and incrementsThe extra year is mostly fulls, which cost far more eachCount of retained restore points by type at the new depth
Deduplication across the windowDistant restore points share fewer blocks than adjacent onesStored size of same type restore points at increasing separations
Chain structure and block generationBlocks stay referenced longer than the policy suggestsHow the software builds and ages chains on the target in use
Immutable copiesLocked versions persist past their nominal expiryLock period applied, and version count held beyond job retention
Restore search spaceMore points to evaluate before choosing oneTime taken to identify a known good point during a drill
Migration at refreshOlder data has to be carried forward or kept readableVolume at each depth and the throughput available to move it

Immutability and the second bill

Immutability changes the character of the decision. A locked object cannot be removed before its retention expires, so a retention extension applied with object lock is a commitment rather than a setting. Under governance mode the hold can be lifted by an authorized identity; under compliance mode it cannot. That difference, set out in governance and compliance modes, determines whether an over long retention can be corrected or only waited out.

The capacity effect is that locked versions accumulate past the point where the backup software would have removed them. Space is not reclaimed on the schedule the policy implies, and the gap between expected and actual consumption grows with the lock period. The shortfall arrives late, once the first cohort of locked versions fails to clear.

The operational effects are quieter but real. A larger retained set enlarges the space to search when choosing a recovery point, which matters most during an incident. It also lengthens the next migration, since data at every depth has to be carried forward or kept readable on the outgoing system; that decision is examined in the discussion of older restore points.

Where ARTESCA fits

ARTESCA is object storage software used as a backup target, deployed on infrastructure the customer runs, across a range of roughly 50 TB to 8.5 PB, with immutability provided through S3 Object Lock and an S3 compatible API. For retention questions the relevant property is that the lock behavior is standard object lock behavior, so version retention can be tested rather than inferred.

Because the deployment sits on infrastructure the organization runs, the capacity consumed by a longer window can be observed directly, in bucket and version terms. That makes the deduplication curve described above measurable in place, using restore points that already exist.

Capacity added to support a longer window arrives as nodes or drives against the existing deployment, so a retention change can be staged against measured consumption instead of being sized in one step. Where retention obligations span multiple sites or reach well beyond that capacity range, RING is the product that addresses it.

Answering the question for one environment

Work the estimate in one direction and write down each step. Count the restore points the new policy holds, by type. Measure the stored size of each type at the depths that already exist. Measure how that size changes as separation increases, and extrapolate along the observed curve rather than a straight line. Add the versions the lock will hold beyond expiry. The result is a range, and a range with its method attached is more defensible than a single figure.

Put the same question to the business alongside the number. An extra year of retention has a cost and a purpose, and the purpose is usually an obligation naming a specific data set. Applying the extension to everything when the obligation covers one application is the most common way retention cost grows without anyone deciding it should.

Finally, revisit the estimate once the first extended cohort has aged into the new window. At that point the deduplication curve, the lock behavior and the restore point count are observations rather than projections, and the model can be corrected before the next extension is requested.

Try ARTESCA free

Immutable object storage that scales from 20TB to petabytes. Deploy a working cluster in under an hour.

Start a free test drive