What is a deduplication appliance?
A deduplication appliance is a purpose-built backup storage system, sold as integrated hardware and software, that reduces stored data by keeping only one copy of each unique block. It is designed to hold long backup retention in much less physical capacity than the logical size of the backups.
How does a deduplication appliance work?
Backup software sends data to the appliance through a file share or a vendor protocol. The appliance divides the stream into chunks, computes a fingerprint for each and stores only chunks it has not seen before. Because successive backups of the same systems are largely identical, most incoming data is already present. The ratio of logical to stored data can be high for repeated full backups and is much lower for data that is already compressed, encrypted or frequently changing.
Where do deduplication appliances fit?
They became common when backups were full copies written daily or weekly and disk was expensive. They still suit certain patterns better than others.
| Situation | Fit |
|---|---|
| Long retention of repeated full backups | Strong: high reduction |
| Database dumps written by native tools | Strong when the dumps are not compressed |
| Incremental-forever backups already reduced at source | Weaker: little left to remove |
| Encrypted or compressed backup streams | Poor: data looks unique |
| Frequent large restores or instant recovery | Limited by rehydration |
What are the trade-offs?
Reading data back requires rehydration, so restores are slower than backups. Many models are fixed in capacity or scale by adding shelves up to a ceiling, after which the unit has to be replaced, a forklift upgrade. The data is held in a proprietary format reachable only through the appliance. And because every backup depends on shared chunks, the integrity of the chunk store is critical. Reading deduplication ratio claims explains how to compare quoted ratios.
How do deduplication appliances handle immutability?
Most current appliances offer a retention lock that prevents backups from being deleted or changed for a set period, and some add isolated copies. The strength of the lock depends on the product and on who can administer the appliance, including access through its management interfaces. When comparing options, the questions are the same as for any immutable backup: who can shorten the lock, what happens to the clock, and whether a separate copy exists.
Frequently asked questions
Is a deduplication appliance the same as a backup appliance?
It is one kind. A backup appliance is any integrated backup hardware. A deduplication appliance is a storage target built around data reduction.
Do deduplication ratios apply to all data?
No. Ratios depend on how repetitive the data is. Compressed, encrypted and already-deduplicated data reduces very little.
Can backup software deduplicate without an appliance?
Yes. Most backup products reduce data at the source or in the repository and can write to general-purpose storage.