NetBackup™ Troubleshooting Guide
- Introduction
- Troubleshooting procedures
- About troubleshooting procedures
- Troubleshooting NetBackup problems
- Troubleshooting installation problems
- Troubleshooting configuration problems
- Device configuration problem resolution
- Testing the master server and clients
- Testing the media server and clients
- Resolving network communication problems with UNIX clients
- Resolving network communication problems with Windows clients
- Troubleshooting vnetd proxy connections
- vnetd proxy connection requirements
- Where to begin to troubleshoot vnetd proxy connections
- Verify that the vnetd process and proxies are active
- Verify that the host connections are proxied
- Test the vnetd proxy connections
- Examine the log files of the connecting and accepting processes
- Viewing the vnetd proxy log files
- Troubleshooting security certificate revocation
- Troubleshooting cloud provider's revoked SSL certificate issues
- Troubleshooting cloud provider's CRL download issues
- How a host's CRL affects certificate revocation troubleshooting
- NetBackup job fails because of revoked certificate or unavailability of CRLs
- NetBackup job fails because of apparent network error
- NetBackup job fails because of unavailable resource
- Master server security certificate is revoked
- Determining a NetBackup host's certificate state
- Troubleshooting issues with external CA-signed certificate revocation
- About troubleshooting networks and host names
- Verifying host name and service entries in NetBackup
- Example of host name and service entries on UNIX master server and client
- Example of host name and service entries on UNIX master server and media server
- Example of host name and service entries on UNIX PC clients
- Example of host name and service entries on UNIX server that connects to multiple networks
- About the bpclntcmd utility
- Using the Host Properties window to access configuration settings
- Resolving full disk problems
- Frozen media troubleshooting considerations
- Troubleshooting problems with the NetBackup web services
- Troubleshooting problems with the NetBackup web server certificate
- Resolving PBX problems
- Troubleshooting problems with validation of the remote host
- Troubleshooting Auto Image Replication
- Troubleshooting network interface card performance
- About SERVER entries in the bp.conf file
- About unavailable storage unit problems
- Resolving a NetBackup Administration operations failure on Windows
- Resolving garbled text displayed in NetBackup Administration Console on a UNIX computer
- Troubleshooting error messages in the NetBackup Administration Console
- Extra disk space required for logs and temporary files for the NetBackup Administration Console
- Unable to logon to the NetBackup Administration Console after external CA configuration
- Troubleshooting file-based external certificate issues
- Troubleshooting Windows certificate store issues
- Troubleshooting backup failures
- Troubleshooting backup failure issues with NAT clients or NAT servers
- Troubleshooting issues with the NetBackup Messaging Broker (or nbmqbroker) service
- Issues with email notifications for Windows systems
- Issues with KMS configuration
- Issues with initiating the NetBackup CA migration because of large key size
- Issues with the non-privileged user (service user) account
- Issues with group name format in the auth.conf file
- Troubleshooting the VxUpdate add package process
- Issues with FIPS mode
- Issues with malware scanning
- Issues with NetBackup jobs that are enabled for data-in-transit encryption
- Issues with Unstructured Data Instant Access
- Using NetBackup utilities
- About NetBackup troubleshooting utilities
- About the analysis utilities for NetBackup debug logs
- About the Logging Assistant
- About network troubleshooting utilities
- About the NetBackup support utility (nbsu)
- About the NetBackup consistency check utility (NBCC)
- About the NetBackup consistency check repair (NBCCR) utility
- About the nbcplogs utility
- About the robotic test utilities
- About the NetBackup Smart Diagnosis (nbsmartdiag) utility
- About log collection by job ID
- Disaster recovery
- About disaster recovery
- About disaster recovery requirements
- Disaster recovery packages
- About disaster recovery settings
- Recommended backup practices
- About disk recovery procedures for UNIX and Linux
- About clustered NetBackup server recovery for UNIX and Linux
- About disk recovery procedures for Windows
- About clustered NetBackup server recovery for Windows
- Generating a certificate on a clustered master server after disaster recovery installation
- About restoring disaster recovery package
- About the DR_PKG_MARKER_FILE environment variable
- Restoring disaster recovery package on Windows
- Restoring disaster recovery package on UNIX
- About recovering the NetBackup catalog
- About NetBackup catalog recovery on Windows computers
- About NetBackup catalog recovery from disk devices
- About NetBackup catalog recovery and symbolic links
- About NetBackup catalog recovery
- NetBackup disaster recovery email example
- About recovering the entire NetBackup catalog
- Establishing a connection with NAT media server before catalog recovery
- About recovering the NetBackup catalog image files
- About recovering the NetBackup relational database
- Recovering the NetBackup catalog when NetBackup Access Control is configured
- Recovering the NetBackup catalog from a nonprimary copy of a catalog backup
- Recovering the NetBackup catalog without the disaster recovery file
- Recovering a NetBackup user-directed online catalog backup from the command line
- Restoring files from a NetBackup online catalog backup
- Unfreezing the NetBackup online catalog recovery media
- Steps to carry out when you see exit status 5988 during catalog recovery
- Index
About log collection by job ID
A new command line interface and API option of gathering relevant logs by specifying a job ID, then upload the gathered logs is included with NetBackup. With the specified job ID, logs within the job execution time frame are gathered from the primary server, media server, and clients if reachable. Legacy logs and try file logs may include logs outside of job execution time frame as those logs do not honor the time duration filter.
A valid job ID must be present in the Activity monitor. By default, a job ID is removed one week after the job is completed. The nblogadm utility cannot gather the logs of a job ID if bpdbjobs or the Activity monitor cannot retrieve the job details of the specified job ID.
The gathered logs include NetBackup product and NetBackup support utility (nbsu) logs. The log gathering supports one record ID at a time, no concurrent log gathering from multiple record IDs.
To avoid filling up the file system on primary server, media server, and client during log gathering, Veritas recommends that you use the KEEP_LOGS_SIZE_GB option. Veritas recommends that you specify the size of NetBackup logs that are retained before you gather the logs. See the NetBackup Administrator's Guide, Volume I for more information.
To avoid the gathered logs filling up the file system on a primary server, a predefined 10GB free space watermark is used. NetBackup uses this watermark to check and prevent the start of log gathering when the available disk space is less than the watermark. Additionally, the gathering process is stopped during logs gathering when the available space on a primary server falls under the watermark. To reduce the free space watermark to 5GB, set the HIGH_WATERMARK_TRB_LOG_RECORDS = 5 in bp.conf file.
To gather logs with higher verbosity level, manually enable logging and configure the desired logging level as documented in the NetBackup Logging Reference Guide. Then restart the job and start a log gathering task.
The gathered logs are stored on the primary server in the directory shown. All contents in the directory are uploaded after you start a log uploading task. Confirm that only the intended files are present in the directory:
Linux and UNIX
/usr/openv/netbackup/logs/nblastaging/record ID-timestamp: YYYYMMDD-HHMMSS
Windows
install_path\Veritas\NetBackup\logs\nblastaging\record ID-timestamp: YYYYMMDD-HHMMSS
Supported job types:
Backup
Backup from Snapshot
Snapshot
Supported workload types:
File System
NDMP (logs are only collected from Primary and media servers)
Oracle (logs are only collected from the Primary server)
Snapshot Manager (logs are only collected from Primary and media servers)
VMware
Unsupported configurations:
Microsoft Cluster Server (MSCS)
VMware access host
You can gather logs from distributed workloads where multiple clients are involved. You must manually gather logs for every job in the Activity monitor where the specific client or node is displayed in the Client column. You must then consolidate all logs. Examples of distributed workloads include Oracle RAC and MSSQL Availability Groups.
The gathered logs can be uploaded to the Veritas Technical Support organization with the command line interface and the API options. See https://www.veritas.com/support/en_US/article.100038665 for more details.
The password that is provided to upload the logs is stored in the form of credential object in the NetBackup credential management pane. It is removed after logs are uploaded. The name of the credential object may be shown briefly during upload, but not the password itself.
Table: New command line interface flags introduced to nblogadm utility
Command line interface | Description |
|---|---|
nblogadm --action getactivecollections --json | Get the number of records that are in-progress. (Does not collect logs for more than one record ID at a time) |
nblogadm --action createrecord --jobid job ID --json | Take a job ID, create an empty log record, and return the created record ID. |
nblogadm --action collectlogsforjob --recid record ID --runnbsu --json | Create a task to gather the logs for the specified record ID. |
nblogadm --action deleterecord --recid record ID --json | Delete the collected logs and record for the specified record ID. This action also terminates any in-progress task. |
nblogadm --action casedetail --recid record ID --json | Get the log gather and the log upload task details for the specified record ID. |
Table: New NetBackup APIs
API | Description |
|---|---|
GET /troubleshooting/log-records | Get the number of records that are in-progress. (Does not collect logs for more than one record ID at a time) |
POST /troubleshooting/log-records | Take a job ID, create an empty log record, and return the created record ID. |
POST /troubleshooting/log-records/record ID/collect | Create a task to gather the logs for the specified record ID. |
POST /troubleshooting/log-records/record ID/upload | Create a task to upload the logs for the specified record ID and SFTP server access information. |
DELETE /troubleshooting/log-records/record ID | Delete the collected logs and record for the specified record ID. This action also terminates any in-progress task. |
GET /troubleshooting/log-records/record ID | Get the log gather and the log upload task details for the specified record ID. |