NetBackup™ Troubleshooting Guide
- Introduction
- Troubleshooting procedures
- About troubleshooting procedures
- Troubleshooting NetBackup problems
- Troubleshooting installation problems
- Troubleshooting configuration problems
- Device configuration problem resolution
- Testing the master server and clients
- Testing the media server and clients
- Resolving network communication problems with UNIX clients
- Resolving network communication problems with Windows clients
- Troubleshooting vnetd proxy connections
- vnetd proxy connection requirements
- Where to begin to troubleshoot vnetd proxy connections
- Verify that the vnetd process and proxies are active
- Verify that the host connections are proxied
- Test the vnetd proxy connections
- Examine the log files of the connecting and accepting processes
- Viewing the vnetd proxy log files
- Troubleshooting security certificate revocation
- Troubleshooting cloud provider's revoked SSL certificate issues
- Troubleshooting cloud provider's CRL download issues
- How a host's CRL affects certificate revocation troubleshooting
- NetBackup job fails because of revoked certificate or unavailability of CRLs
- NetBackup job fails because of apparent network error
- NetBackup job fails because of unavailable resource
- Master server security certificate is revoked
- Determining a NetBackup host's certificate state
- Troubleshooting issues with external CA-signed certificate revocation
- About troubleshooting networks and host names
- Verifying host name and service entries in NetBackup
- Example of host name and service entries on UNIX master server and client
- Example of host name and service entries on UNIX master server and media server
- Example of host name and service entries on UNIX PC clients
- Example of host name and service entries on UNIX server that connects to multiple networks
- About the bpclntcmd utility
- Using the Host Properties window to access configuration settings
- Resolving full disk problems
- Frozen media troubleshooting considerations
- Troubleshooting problems with the NetBackup web services
- Troubleshooting problems with the NetBackup web server certificate
- Resolving PBX problems
- Troubleshooting problems with validation of the remote host
- Troubleshooting Auto Image Replication
- Troubleshooting network interface card performance
- About SERVER entries in the bp.conf file
- About unavailable storage unit problems
- Resolving a NetBackup Administration operations failure on Windows
- Resolving garbled text displayed in NetBackup Administration Console on a UNIX computer
- Troubleshooting error messages in the NetBackup Administration Console
- Extra disk space required for logs and temporary files for the NetBackup Administration Console
- Unable to logon to the NetBackup Administration Console after external CA configuration
- Troubleshooting file-based external certificate issues
- Troubleshooting Windows certificate store issues
- Troubleshooting backup failures
- Troubleshooting backup failure issues with NAT clients or NAT servers
- Troubleshooting issues with the NetBackup Messaging Broker (or nbmqbroker) service
- Issues with email notifications for Windows systems
- Issues with KMS configuration
- Issues with initiating the NetBackup CA migration because of large key size
- Issues with the non-privileged user (service user) account
- Issues with group name format in the auth.conf file
- Troubleshooting the VxUpdate add package process
- Issues with FIPS mode
- Issues with malware scanning
- Issues with NetBackup jobs that are enabled for data-in-transit encryption
- Issues with Unstructured Data Instant Access
- Using NetBackup utilities
- About NetBackup troubleshooting utilities
- About the analysis utilities for NetBackup debug logs
- About the Logging Assistant
- About network troubleshooting utilities
- About the NetBackup support utility (nbsu)
- About the NetBackup consistency check utility (NBCC)
- About the NetBackup consistency check repair (NBCCR) utility
- About the nbcplogs utility
- About the robotic test utilities
- About the NetBackup Smart Diagnosis (nbsmartdiag) utility
- About log collection by job ID
- Disaster recovery
- About disaster recovery
- About disaster recovery requirements
- Disaster recovery packages
- About disaster recovery settings
- Recommended backup practices
- About disk recovery procedures for UNIX and Linux
- About clustered NetBackup server recovery for UNIX and Linux
- About disk recovery procedures for Windows
- About clustered NetBackup server recovery for Windows
- Generating a certificate on a clustered master server after disaster recovery installation
- About restoring disaster recovery package
- About the DR_PKG_MARKER_FILE environment variable
- Restoring disaster recovery package on Windows
- Restoring disaster recovery package on UNIX
- About recovering the NetBackup catalog
- About NetBackup catalog recovery on Windows computers
- About NetBackup catalog recovery from disk devices
- About NetBackup catalog recovery and symbolic links
- About NetBackup catalog recovery
- NetBackup disaster recovery email example
- About recovering the entire NetBackup catalog
- Establishing a connection with NAT media server before catalog recovery
- About recovering the NetBackup catalog image files
- About recovering the NetBackup relational database
- Recovering the NetBackup catalog when NetBackup Access Control is configured
- Recovering the NetBackup catalog from a nonprimary copy of a catalog backup
- Recovering the NetBackup catalog without the disaster recovery file
- Recovering a NetBackup user-directed online catalog backup from the command line
- Restoring files from a NetBackup online catalog backup
- Unfreezing the NetBackup online catalog recovery media
- Steps to carry out when you see exit status 5988 during catalog recovery
- Index
Troubleshooting file-based external certificate issues
This issue may occur because of one of the following reasons:
The web service certificate that is used for communication is not configured properly.
Some of the NetBackup core services have not started.
The required prerequisites for external certificate are not met.
External certificate configuration path (ECA_CERT_PATH) is not configured properly.
Certificate revocation check failed.
To resolve the issue, review the following causes and run the following command to determine the current state of the problem.
Install_Path/bin/nbcertcmd -enrollCertificate -preCheck -server server_name
Install_Path refers to the following:
On Windows: VERITAS\NetBackup\bin
On Unix: /usr/openv/netbackup/bin
The NetBackup web server is not configured to use external certificates.
The following error is displayed:
EXIT STATUS 26: client/server handshaking failed.
Run the following command on the master server to check if external CA is configured (ON) or not (OFF).
Install_Path/nbcertcmd -getSecConfig -caUsage
On Windows: C:\Program Files\ VERITAS\NetBackup\bin\nbcertcmd -getSecConfig -caUsage
On Unix: /usr/openv/netbackup/bin/netbackup/bin/nbcertcmd -getSecConfig -caUsage
For example: C:\Program Files\Veritas\NetBackup\bin>nbcertcmd -getSecConfig -caUsage
Output:
NBCA:OFF ECA:ON
If an external CA is not configured, run the configureWebServerCerts command on the web server.
In certain cases, you may also get the following error when an external CA is not configured on the web server.
EXIT STATUS 5982: The certificate revocation list is unavailable.
In this case, first check the value of the ECA parameter. If it is OFF, run the configureWebServerCerts command.
The web service certificate that is used for communication is not trusted by a certificate authority.
Check the certificate path (the configureWebServerCert -certPath option) must have a leaf certificate with the entire chain of CA certificates except the trust anchor (root CA).
Run the following command to list the certificates that are configured for the web server.
nbcertcmd -listallcertificates -jks
On Windows: C:\Program Files\ VERITAS\NetBackup\bin\nbcertcmd -listallcertificates -jks
On Unix: /usr/openv/netbackup/bin/netbackup/bin/nbcertcmd -listallcertificates -jks
Run the following command to list the host certificate details of the NetBackup master server.
Install_Path/goodies/nbsslcmd x509 -in certificate_path -noout -text -purpose
On Windows: C:\Program Files\ VERITAS\NetBackup\bin\goodies\nbsslcmd x509 -in certificate_path -noout -text -purpose
On Unix: /usr/openv/netbackup/bin/netbackup/bin/goodies/nbsslcmd x509 -in certificate_path -noout -text -purpose
Validate whether the host certificate of the master server is issued by the same root CA as of the web server certificate.
If host certificate is not issued by the same root CA as of web server certificate then issue new certificate with that CA for NetBackup Master server and enroll certificate again.
The specified server name was not found in the web service certificate.
The server name does not match any of the host names listed in the server's certificate.
Names listed in the server's certificate are:
DNS: nb-master _ext
DNS: nb-master .some.domain.com
DNS: nb-master _web_svr EXIT STATUS 8509:
Either update the configuration on the NetBackup host so that it uses one of the names that are present in the web server certificate to refer to the master server or Include all names of the master server that are known to the NetBackup domain in the certificate.
For more information, refer to the following article:
https://www.veritas.com/support/en_US/article.000126751
Some of the NetBackup core services have not started.
Carry out the following procedure to resolve the issue:
Check the status of the following services by running the bpps command from the NetBackup/bin directory:
nbsl
vnetd -standalone
NB_dbsrv (on UNIX) or the dbsrv16 (on Windows)
For more details on the NetBackup commands, refer to the NetBackup Commands Reference Guide.
Start the nbsl and the vnetd services, if they are not running.
Start the NB_dbsrv (on Unix) service or the dbsrv16 (on Windows) service, if it is not running.
Restart nbsl, vnetd, and NB_dbsrv (or dbsrv16) services as follows:
On Windows:
Install_Path\bin\bpdown -e "NetBackup Service Layer" -f -v
Install_Path\bin\bpup -e "NetBackup Service Layer" -f -v
Install_Path\bin\bpdown -e "NetBackup Legacy Network Service" -f -v
Install_Path\bin\bpup -e "NetBackup Legacy Network Service" -f -v
Install_Path\bin\bpdown -e "SQLANYs_VERITAS_NB" -f -v
Install_Path\bin\bpup -e "SQLANYs_VERITAS_NB" -f -v
Alternatively, you may use the Service Control Manager to restart the NetBackup Service Layer (NBSL), NetBackup Legacy Network Service (vnetd) , and SQLANYs_VERITAS_NB services.
For example:
C:\Program Files\Veritas\NetBackup\bin\bpdown -e "NetBackup Service Layer" -f -v
C:\Program Files\Veritas\NetBackup\bin\bpup -e "NetBackup Service Layer" -f -v
C:\Program Files\Veritas\NetBackup\bin\bpdown -e "NetBackup Legacy Network Service" -f -v
C:\Program Files\Veritas\NetBackup\bin\bpup -e "NetBackup Legacy Network Service" -f -v
C:\Program Files\Veritas\NetBackup\bin\bpdown -e "SQLANYs_VERITAS_NB" -f -v
C:\Program Files\Veritas\NetBackup\bin\bpup -e "SQLANYs_VERITAS_NB" -f -v
On Unix:
Install_Path/netbackup/bin/nbsl -terminate
Install_Path/netbackup/bin/nbsl
To stop vnetd and NB_dbsrv, refer to the following example:
To start vnetd and NB_dbsrv, run the following commands:
install_path/netbackup/bin/vnetd -standalone install_path/db/bin/NB_dbsrv
For example:
/usr/openv/netbackup/bin/nbsl -terminate
/usr/openv/netbackup/bin/nbsl
# ps -fed | grep vnetd | grep standalone
root 16018 1 4 08:47:35 ? 0:01 ./vnetd -standalone
# kill 16018
# ps -fed |grep NB_dbsrv
root 11959 1 4 08:47:35 ? 0:01 ./NB_dbsrv
root 16174 16011 0 08:47:39 pts/2 0:00 grep ./NB_dbsrv
# kill 11959
/usr/openv/netbackup/bin/vnetd -standalone
/usr/openv/db/bin/NB_dbsrv
If the problem persists, contact the Technical Support team.
The required prerequisites for external certificate are not met.
Review the following prerequisites:
Subject DN should be unique and stable for each host. It should have less than 255 characters and should not be empty.
Only ASCII 7 characters are supported in the certificate subject DN and X509v3 Subject Alternative Name.
Server and client authentication attributes (SSL server and SSL client) should be set (or should be true) in the certificate.
Certificate is in PEM format.
CRL distribution points (CDPs) are supported only for HTTP/HTTPS.
Run the following command to verify if the prerequisites are met.
Install_Path/goodies/nbsslcmd x509 -in certificate_path -noout -text -purpose
Note:
The certificate paths that are provided for the configureWebServerCert -certPath option and the ECA_CERT_PATH option must have a leaf certificate with the entire chain of the CA certificates except the trust anchor (root CA).
Desirable conditions:
Host name (CLIENT_NAME) that is used for certificate enrollment should be part of X509v3 Subject Alternative Name under DNS type.
Common name (CN) of the subject name should not be empty.
Note:
The following warning is generated when the nbsslcmd command is run and can be safely ignored:
WARNING: can't open config file: /usr/local/ssl/openssl.cnf
External certificate configuration path is not configured properly.
Ensure the following external certificate configuration options are configured properly:
ECA_CERT_PATH
ECA_TRUST_STORE_PATH
ECA_PRIVATE_KEY_PATH
ECA_CRL_PATH
ECA_CRL_CHECK
Ensure the following:
The peer host certificate has the CRL distribution point (CDP).
If you have not specified ECA_CRL_PATH, NetBackup uses the CRLs on the URLs that are specified in the peer host certificate's CDP.
ECA_CRL_PATH is not a volumeID path on Windows.
Run the following command and validate the external certificate configuration parameters.
On UNIX: Install_Path/bin/nbgetconfig | grep ECA
Windows: Install_Path/bin/nbgetconfig | findstr ECA
.
For more information about the configuration options, refer to the NetBackup Security and Encryption Guide.
The requirements that are mentioned in Cause 3 are not met.
Host name (CLIENT_NAME) used for the certificate enrollment is not part of X509v3 Subject Alternative Name under the DNS type.
If enrollment fails with this error, do one of the following:
Generate new certificate having host name in subject alternative name of the certificate.
Add or update (first delete and then add) the subject name of the certificate (RFC 2253 compliant) in the external certificate database on the master server.
Run the following command to add an entry for the host and the associated subject name in the NetBackup certificate database (only administrator can perform this operation):
Install_Path/bin/nbcertcmd -createECACertEntry -host host_name | -hostId host_id -subject subject name of external cert [-server master_server_name]
Alternatively, run the following command to delete an entry for the host and the associated subject name from the NetBackup certificate database and then add an entry using the -createECACertEntry command (only administrator can perform this operation):
Install_Path/bin/nbcertcmd -deleteECACertEntry -subject subject name of external cert [-server master_server_name]
Common name (CN) of the subject name is not present in the certificate.
If certificate enrollment fails with this error, do one of the following:
Generate a new certificate with the common name in the certificate.
Generate a new certificate with the host name in the subject alternative name of the certificate.
Add host in the NetBackup host database and add an entry for the host and the associated subject name in the NetBackup certificate database.
Run the following command to add a host in the NetBackup host database (only administrator can perform this operation):
Install_Path/bin/admincmd/nbhostmgmt -addhost -host host_name | -hostId host_id [-server master_server_name]
Run the following command to add an entry for the host and the associated subject name in the NetBackup certificate database.
Install_Path/bin/nbcertcmd -createECACertEntry -host host_name | -hostId host_id -subject subject name of external cert [-server master_server_name]
Subject name of the external certificate should be RFC 2253 compliant.
Certificate revocation check failed.
External certificate enrollment can fail with the certificate revocation error for the following reasons:
The external certificate is revoked.
The web server certificate is revoked.
CRL is unavailable on either the host or the master server.
See Troubleshooting issues with external CA-signed certificate revocation.
For more details on enrollment of external certificates in NetBackup, refer to the NetBackup Security and Encryption Guide.