Veritas NetBackup™ Troubleshooting Guide

Step 1

Create debug log directories.

During communication retries, the debug logs provide detailed debug information, which can help you analyze the problem.

Use the bprd log directory to debug client to master server communication, not client to media server communication problems.

Step 2

Test a new configuration or modified configuration.

Step 3

Verify name resolution.

To verify name resolution, run the following command on the master server and the media servers:

# bpclntcmd -hn client name

If the results are unexpected, review the configuration of these name resolution services: nsswitch.conf file, hosts file, ipnodes file, and resolv.conf file.

Also run the following on the client to check forward and reverse name lookup of the master server and media server that perform the backup:

# bpclntcmd -hn server name

# bpclntcmd -ip IP address of server

Step 4

Verify network connectivity.

Verify network connectivity between client and server by pinging the client from the server.

# ping clientname

Where clientname is the name of the client as configured in the NetBackup policy configuration.

For example, to ping the policy client that is named ant:

# ping ant 
    ant.nul.nul.com: 64 byte packets
    64 bytes from 199.199.199.24: icmp_seq=0. time=1. ms
    ----ant.nul.nul.com PING Statistics----
    2 packets transmitted, 2 packets received, 0% packet 
    loss round-trip (ms)  min/avg/max = 1/1/1

A successful ping verifies connectivity between the server and client. If the ping fails and ICMP is not blocked between the hosts, resolve the network problem outside of NetBackup before you proceed.

Some forms of the ping command let you ping the bpcd port on the client as in the following command:

# ping ant 1556     
    

Ping 1556 (PBX) and 13724 (vnetd) in sequence, the same sequence that NetBackup tries by default. You then know which ports are closed so that you can open them for more efficient connection tries.

Step 5

Ensure that the client listens on the correct port for the bpcd connections.

On the client, run one of the following commands (depending on platform and operating system):

netstat -a | grep bpcd
netstat -a | grep 13782
rpcinfo -p | grep 13782

Repeat for 1556 (PBX) and 13724 (vnetd). If no problems occur with the ports, the expected output is as follows:

# netstat -a | egrep '1556|PBX|13724|vnetd|13782|bpcd' | grep LISTEN
 *.1556          *.*      0      0 49152      0 LISTEN
 *.13724         *.*      0      0 49152      0 LISTEN
 *.13782         *.*      0      0 49152      0 LISTEN

LISTEN indicates that the client listens for connections on the port.

If the NetBackup processes are running correctly, the expected output is as follows:

# ps -ef | egrep 'pbx_exchange|vnetd|bpcd' | grep -v grep
root   306 1 0 Jul 18 ? 13:52 /opt/VRTSpbx/bin/pbx_exchange
root 10274 1 0 Sep 13 ?  0:11 /usr/openv/netbackup/bin/vnetd -standalone
root 10277 1 0 Sep 13 ?  0:45 /usr/openv/netbackup/bin/bpcd -standalone

Repeat the procedure on the master server(s) and media server(s), to test communication to the client.

Step 6

Connect to the client through telnet.

On the client, telnet to 1556 (PBX) and 13724 (vnetd). Check both ports to make sure that a connection is made on at least one of them. If the telnet connection succeeds, keep the connection until after you perform step 8, then terminate it with Ctrl-c.

telnet clientname 1556
telnet clientname 13724

Where clientname is the name of the client as configured in the NetBackup policy configuration.

For example,

# telnet ant vnetd
Trying 199.999.999.24 ...
Connected to ant.nul.nul.com.
Escape character is '^]'.

In this example, telnet can establish a connection to the client ant.

Repeat the procedure on the master server(s) and media server(s), to test communication to the client.

Step 7

Identify the outbound socket on the server host.

On the master server(s) and media server(s): Use the following command to identify the outbound socket that is used for the telnet command from step 6. Specify the appropriate IP address to which the server resolves the policy client. Note the source IP (10.82.105.11), the source port (45856) and the destination port (1556).

# netstat -na | grep '<client_IP_address>' | egrep '1556|13724' 
10.82.105.11.45856 10.82.104.99.1556 49152 0 49152 0 ESTABLISHED

If telnet is still connected and a socket is not displayed: Remove the port number filtering and observe the port number to which the site has mapped the service name. Check that process listens on the port number in step 5.

$ netstat -na | grep '<client_IP_address>' 
10.82.105.11.45856 10.82.104.99.1234 49152 0 49152 0 ESTABLISHED

If the socket is in a SYN_SENT state instead of an ESTABLISHED state, the server host is trying to make the connection. However, a firewall blocks the outbound TCP SYN from reaching the client host or blocks the return bound TCP SYN+ACK from reaching the server host.

Step 8

Confirm that the telnet connection reaches this client host.

On the master server(s) and media server(s), to confirm that the telnet connection reaches this client host, run the following command:

$ netstat -na | grep '<source_port>'
10.82.104.99.1556  10.82.105.11.45856  49152 0 49152 0 ESTABLISHED

Step 9

Verify communication between the client and the master server.

To verify client to master server communications, use the bpclntcmd utility. When -pn and -sv run on a NetBackup client, they initiate inquiries to the NetBackup master server (as configured in the client bp.conf file). The master server then returns information to the requesting client. More information is available about bpclntcmd.

See About the bpclntcmd utility.

The PBX, vnetd, and bprd debug logs should provide details on the nature of any remaining failure.