Search <book_title>...

NetBackup™ Deduplication Guide

Last Published: 2022-03-28

Product(s): NetBackup (10.0)

About the two modes of the Encryption Crawler

The Encryption Crawler is not turned on by default. You must explicitly enable it with the crcontrol command. Encryption Crawler has two modes: Graceful mode and Aggressive mode. These two modes can have an effect on how certain jobs perform. Review the following information to help you select the right mode for your environment.

Graceful mode

Unless the user specifies a different mode with the crcontrol --encconvertlevel command, Encryption Crawler's default mode is Graceful. In this mode, it runs only when the MSDP pool is relatively idle and no compaction or CRQP jobs are active. It usually means no backup, restore, duplication, or replication jobs are active on the MSDP pool when the MSDP pool is idle. To prevent Encryption Crawler from overloading the system it doesn't run continuously. When the Encryption Crawler is in Graceful mode, it may take a longer time to finish.

The Graceful mode checks that the MSDP pool is relatively idle. It checks the pool state by calculating the I/O statistics on the MSDP pool and checks that no compaction or CRQP jobs are active before it processes each data container. It pauses if the MSDP pool is not idle, compaction, or CRQP jobs are active. In most cases, Graceful mode pauses when backup, restore, duplication, or replication jobs are active on the MSDP pool.

If the data deduplication rate of the active NetBackup jobs is high, the I/O operation rate could be low and the MSDP pool could be relatively idle. In this case, the Graceful mode may run if no compaction or CRQP jobs are active.

If the MSDP fingerprint cache loading is in progress, the I/O operation rate on the MSDP pool is not low. In this case, the Graceful mode may pause and wait for the fingerprint cache loading to finish. The Encryption Crawler monitors the spoold log and waits for the message that begins with ThreadMain: Data Store nodes have completed cache loading before restarting. The location of the spoold log is: storage_path/log/spoold/spoold.log. To check if compaction or CRQP jobs are active, run the crcontrol --compactstate or crcontrol --processqueueinfo command.

To have the Graceful mode run faster, you can use the Advanced Options of CheckSysLoad, BatchSize, and SleepSeconds to tune the behavior and performance of Graceful mode. With a larger number for BatchSize and a smaller number for SleepSeconds, Graceful mode runs more continuously.

If you turn off CheckSysLoad, Graceful mode runs while backup, restore, duplication, replication, compaction, or CRQP jobs are active. Such changes can make Graceful mode more active, however it's not as active as Aggressive mode.

Aggressive mode

In this mode, the Encryption Crawler disables CRC check and compaction. It runs while backup, restore, duplication, replication, or CRQP jobs are active.

The Aggressive mode affects the performance of backup, restore, duplication, and replication jobs. To minimize the effect, use the Graceful mode. This choice pauses the encryption process while the system is busy but slows down the encryption process. The Aggressive mode keeps the process active and aggressively running regardless of system state.

The following points are items to consider when Aggressive mode is active:

Any user inputs and the last progress are retained on MSDP restart. You don't need to re-run the command again to recover. The Encryption Crawler recovers and continues from the last progress automatically.
You must enforce encryption with the encrypt keyword on the ServerOptions option in the contentrouter.cfg file in MSDP. You must also restart MSDP before enabling Encryption Crawler, otherwise the Encryption Crawler does not indicate that it is enabled.
If your environment is upgraded from a release older than NetBackup 8.1, you must wait until the rolling Data Conversion finishes before you enable the Encryption Crawler. If you don't wait, the Encryption Crawler does not indicate that it is enabled.
You cannot repeat the Encryption Crawler process after it finishes. Only the data that existed before you enable encryption is unencrypted. All the new data is encrypted inline and does not need the scanning and crawling.
If you disable encryption enforcement after the Encryption Crawler process finishes, the Encryption Crawler state is reset. You can restart the Encryption Crawler process when encryption is enforced again. The time that is required to finish depends on the following items:
- How much new and unencrypted data is ingested.
- How much data resides on the MSDP pool.

Resource utilization for the Graceful and Aggressive modes

Memory: The Encryption Crawler can consume an additional 1 GB of memory for each MSDP partition. The Graceful mode consumes less memory than the Aggressive mode.

CPU: The major CPU utilization by the Encryption Crawler is by the data encryption with AES-256-CTR algorithm. The CPU utilization is less than backing up the same quantity of data. During the process, there is no fingerprinting, inter-component, or inter-node data transfer happening.

Disk I/O: The Encryption Crawler is I/O intensive especially in the Aggressive mode. The Aggressive mode competes for I/O significantly with the active jobs, and it may commit more I/O than the backup jobs.