Emerging Technologies Understanding Deduplication Kevin Carpenter Account Manager Upstate NY Phil Benincasa System Engineer Upstate NY

Emerging Technologies Agenda Introduction What is Deduplication? Why Deduplicate Data? Types of Dedupe Solutions Questions…

Emerging Technologies What is De-duplication? Deduplication is the process of removing redundant data from any storage medium based upon identifying repeating components either prior to or shortly after writing to the media. Deduplication is used primarily on Backup and Archived data as this data contains the most redundancy, is the least performance sensitive data, and consumes the most media capacity in a data center. Current deduplication technology focuses on block level redundancy of data because it provides high redundancy with a manageable deduplication process.

Emerging Technologies Why Deduplicate Data? Week 1 Backup Week 2 Backup Week 3 Backup Normal Disk Store Tape Copy for Vaulting or Offsite 10X Reduction preserved Rehydrate  Backup and Archive copies by nature have lots of redundant contents – common block patterns inside DBs, images, file versions, etc.  Eliminate redundant block patterns across file, databases, images and your archive = retain more backup copies on disk 90%+ reduction in disk usage  Data can’t spin for ever…extend and preserve that reduction benefit to offline storage to support retention, vaulting or offsite DR needs 90%+ reduction in tape usage

Emerging Technologies Why Deduplicate Data?  Data is growing at an alarming rate. Faster than hardware capacity is growing  Less Disk and Tape to consume and manage  Less power and cooling in the data center  Preserve current infrastructure (space and hardware)  Less people required to manage more data $$$$

Emerging Technologies Types of Dedupe Solutions All Deduplication works using the same process!!!! The only difference is where the steps occur.

Emerging Technologies Types of Dedupe Solutions All processing occurs within the system. The dedupe database and storage are contained within the unit Data is deduplicated either in-line or lands in a common storage area and is then deduped post-write. Appliance Based Advantages Easy to acquire Dedicated Hardware and to the processing Flexible deployment options Disadvantages Do not scale easily Limited scope of de-duplication Requires more data to flow through backup and archive process Ties you into a specific solution Can become a performance bottleneck

Emerging Technologies Software Based Advantages Lower cost of acquisition More efficient in limiting data flow Easier to manage Balances performance and processing over entire process Free to choose whatever hardware makes sense Disadvantages More complex to license Tied to a specific software package Requires hardware for performance that is not all-in-one Types of Dedupe Solutions

Emerging Technologies Client Side Dedupe Advantages Most efficient in limiting data flow Best overall performance in backup environments Free to choose whatever hardware makes sense Easier to manage than Appliance Disadvantages More complex to license Tied to a specific software package Requires hardware for performance that is not all-in-one Heavier processing requirements on clients Types of Dedupe Solutions

Emerging Technologies Questions…. ??????

Emerging Technologies Thank You For Your Time