Beth on SearchStorage.com started this great thread and I wanted to comment on it on my own home turf, as it were.
Data DeDuplication. Also known as compression, hasn’t changed since the early days of PKZIP 1.0.
Compression works by identifying like blocks of data and replacing them with a single block and pointers to every place the block was found. One of the main reasons it works so well in plain text applications is that there are only so many combinations of ascii characters that can be found.
I find it interesting that this seemingly old technology has found new life in the form of the seemingly complicated “Data DeDuplication”.
So far – no one has sufficiently explained to me the benefits of using a Data Deduplication product over the conventional in-band tape compression. Obviously offloading compression to something with a real processor might gain you some performance and maybe even allow compression to happen without causing a tape to ‘shoe-shine’ across the head as it keeps having to back up. However I have not yet seen a single example that justifies the cost and effort involved.