Veritas NetBackup™ Deduplication Guide
- Introducing the NetBackup media server deduplication option
- Planning your deployment
- About MSDP storage and connectivity requirements
- About NetBackup media server deduplication
- About NetBackup Client Direct deduplication
- About MSDP remote office client deduplication
- About MSDP performance
- MSDP deployment best practices
- Provisioning the storage
- Licensing deduplication
- Configuring deduplication
- Configuring the Deduplication Multi-Threaded Agent behavior
- Configuring the MSDP fingerprint cache behavior
- Configuring MSDP fingerprint cache seeding on the storage server
- About MSDP Encryption using KMS service
- Configuring a storage server for a Media Server Deduplication Pool
- Configuring a disk pool for deduplication
- Configuring a Media Server Deduplication Pool storage unit
- About MSDP optimized duplication within the same domain
- Configuring MSDP optimized duplication within the same NetBackup domain
- Configuring MSDP replication to a different NetBackup domain
- About NetBackup Auto Image Replication
- Configuring a target for MSDP replication to a remote domain
- Creating a storage lifecycle policy
- Resilient Network properties
- Editing the MSDP pd.conf file
- About protecting the MSDP catalog
- Configuring an MSDP catalog backup
- Configuring deduplication to the cloud with NetBackup CloudCatalyst
- Using NetBackup CloudCatalyst to upload deduplicated data to the cloud
- Configuring a CloudCatalyst storage server for deduplication to the cloud
- Monitoring deduplication activity
- Viewing MSDP job details
- Managing deduplication
- Managing MSDP servers
- Managing NetBackup Deduplication Engine credentials
- Managing Media Server Deduplication Pools
- Changing a Media Server Deduplication Pool properties
- Configuring MSDP data integrity checking behavior
- About MSDP storage rebasing
- Managing MSDP servers
- Recovering MSDP
- Replacing MSDP hosts
- Uninstalling MSDP
- Deduplication architecture
- About unified logging
- About legacy logging
- Troubleshooting MSDP installation issues
- Troubleshooting MSDP configuration issues
- Troubleshooting MSDP operational issues
- Troubleshooting CloudCatalyst issues
- CloudCatalyst logs
- Problems encountered while using the Cloud Storage Server Configuration Wizard
- Disk pool problems
- Problems during cloud storage server configuration
- CloudCatalyst troubleshooting tools
- Appendix A. Migrating to MSDP storage
About MSDP fingerprinting
NetBackup uses a unique identifier to identify each file and each file segment that is backed up. The deduplication plug-in reads the backup image and separates the image into files. The plug-in separates the files into segments. For each segment, the plug-in calculates the hash key (or fingerprint) that identifies each data segment. To create a hash, every byte of data in the segment is read and added to the hash.
NetBackup 8.0 and previous versions use fingerprinting calculations that are based on the MD5-like algorithm. Starting with NetBackup 8.1, the fingerprinting calculations are based on a more secure SHA-2 algorithm. On a system that is upgraded to the 8.1 version, every new segment is computed with the SHA-2 algorithm. A data rolling conversion task works in the background to convert the existing MD5-like fingerprints to SHA-2 fingerprints, gradually.
See About the rolling data conversion mechanism for MSDP.
NetBackup 8 .1 can handle both fingerprint types, and the new server is compatible with old clients and old servers. When you perform a backup from an old client to a new server or when you duplicate data from an old server to a new server, conversion from MD5-like to SHA-2 occurs inline on the new server before the data is saved to the disk. Similarly, when you duplicate data from a new server to an old server, conversion from SHA-2 to MD5-like occurs inline on the new server before the data is sent to the old server.
Notes and restrictions that there are some known issues for the compatibility support.
The fingerprint conversion requires additional computation time. The interaction between old clients and old servers and new server is slower than if both the client and the server are new.
You cannot restore data that is backed up using SHA-2 algorithm on a media server that uses the MD5-like algorithm. However, you may choose to restore the SHA-2 fingerprint data on a new media server.
Similarly, you cannot use client-direct restore to restore data that is backed up using Client Direct deduplication on a media server that uses the MD5-like algorithm. However, you may choose to restore the data on a new media server.
If you are using two types of media servers for load balancing, where one media server uses MD5-like algorithm and the other media server uses the SHA-2 algorithm, the initial backup may lose deduplication ratio. Therefore, split the old media servers and the new media servers into different groups, and create different storage unit for each of them.
When data is backed up from a NetBackup 7.5 or previous version client, using Client Direct deduplication, most of the data is transferred over the network and deduplicated on the server. This may save storage, but it does not reduce network throughput. It is recommended that you upgrade the NetBackup client to the latest version.