Veritas NetBackup™ Status Codes Reference Guide

• VSS_E_BAD_STATE snapshot error

• VSS_E_INSUFFICIENT_STORAGE snapshot error

• NetBackup cannot obtain the volume ID of a drive

  NetBackup may not be able to obtain the volume ID of a drive. In that case, none of the virtual machine drives are backed up. The backup fails with NetBackup status code 156.

  The drive may be down.

• A backup of the virtual machine is already active

  You cannot run more than one backup per virtual machine at a time. If you start a second backup of the virtual machine while the first backup is active, the second job fails with a status 156.

  Recommended action: Wait until the first job completes, then run the second one.

• Cannot find virtual machine name

  NetBackup cannot find the host name or VM display name of a virtual machine that is listed in the backup policy. The detailed status log may include the following error message:

  Critical bpbrm (pid=<pid number>) from client <client name>:
  FTL - snapshot creation failed, status 156.)

  If the virtual machines do not have static IP addresses, you can configure NetBackup to identify virtual machines by their VM display names or UUIDs. Examples of the environments that do not use static IP addresses are clusters, and the networks that assign IP addresses dynamically.

  Note that NetBackup may have been configured to identify virtual machines by their VM display names. In that case, make sure that the display names are unique and that they do not contain special characters.

• The virtual machine is turned off

  Through a vCenter server, NetBackup can back up the virtual machines that are turned off. You must provide credentials for NetBackup to access the vCenter server.

  If NetBackup uses credentials for an ESX server instead of vCenter, it may not be able to identify a turned off virtual machine. Note the following:

  • If the policy uses VM host name or VM DNS name as the Primary VM identifier, NetBackup may not find the virtual machine. The backup fails.

  • If the policy uses VM display name or VM UUID as the Primary VM identifier, NetBackup can identify the virtual machine. The backup succeeds.

• The virtual machine has one or more independent disks and is in a suspended state

  If a virtual machine with independent disks is in a suspended state, snapshot jobs fail. Messages similar to the following appear in the job details log:

  01/12/2015 17:11:37 - Critical bpbrm (pid=10144) from client 
  <client name>: FTL - VMware error received: Cannot take a 
  memory snapshot, since the virtual machine is configured with 
  independent disks.

  More information is available in the following VMware article:

  http://kb.vmware.com/kb/1007532

  As a workaround, change the state of the virtual machine to on or off, and rerun the backup.

  Note:

  Data on independent disks cannot be captured with a snapshot. The rest of the virtual machine data is backed up.

• The virtual machine's disk is in raw mode (RDM)

  The RDM is ignored (not backed up) and any independent disk is recreated but empty.

• The attempt to create a snapshot exceeded the VMware timeout

  If the attempt to create a snapshot of the virtual machine exceeds the VMware timeout of 10 seconds, the snapshot fails with NetBackup status 156. This timeout may occur if the virtual machine is configured with a large number of volumes. Note that the timeout may be encountered even if the Virtual machine quiesce option was disabled.

  Do one of the following:

  • Reduce the number of volumes within the virtual machine.

  • Install a NetBackup client on the virtual machine and select another backup method for the policy (not the VMware snapshot method).

• The virtual machine has no vmdk file assigned

  Virtual machines without vmdk files can occur in a vCenter Site Recovery Manager (SRM) environment. If a replicated virtual machine has never been active, it is in passive mode and may have no vmdk file(s).

  You can enable the Ignore diskless VMs option on the VMware Advanced Attributes tab of the policy. If this option is enabled: NetBackup does not back up a replicated (passive) virtual machine in an SRM environment if that virtual machine has no vmdk files.

• The vmdk file has too many delta files

  Whenever a VMware snapshot occurs, a delta.vmdk file is created for each vmdk. If 32 or more such delta files exist for a single vmdk file, a NetBackup backup of that VM may fail (status 156). The NetBackup Activity Monitor job details contain messages similar to the following:

  02/06/2015 10:33:17 - Critical bpbrm (pid=15799) from 
  client fl5vm1_2012: FTL - vSphere_freeze: Unable to proceed with 
  snapshot creation, too many existing delta files(44).
  02/06/2015 10:33:17 - Critical bpbrm (pid=15799) from client
  fl5vm1_2012: FTL - VMware_freeze: VIXAPI freeze (VMware snapshot)
  failed with 25: SYM_VMC_FAILED_TO_CREATE_SNAPSHOT
  02/06/2015 10:33:17 - Critical bpbrm (pid=15799) from client
  fl5vm1_2012: FTL - vfm_freeze: method: VMware_v2, type: FIM,
  function: VMware_v2_freeze

  To back up the VM, do the following:

  • Consolidate the VM's snapshots.

    In the VMware interface, right-click on the VM and select Snapshot > Consolidate. For more information, see your VMware documentation.

  • Verify that each of the VM's vmdk files now has fewer than 32 delta files.

    If the snapshot consolidation was not successful, see the following VMware article for further assistance:

    http://kb.vmware.com/kb/1004545

  • Rerun the NetBackup backup.

• VMware snapshot quiesce operation failed

  If the NetBackup policy is enabled for virtual machine quiesce (the default), the VMware snapshot operation in vSphere initiates a quiesce of the virtual machine. If snapshot quiesce fails, the NetBackup job fails with status 156.

• NetBackup for VMware Administrator's Guide

• NetBackup for Hyper-V Administrator's Guide

• NetBackup Snapshot Client Administrator's Guide

To increase the VSS cache size by using the Shadow Copy configuration in Windows, do the following in the order listed:

• In Windows, right-click My Computer and select Manage.

• In the console tree, right-click Shared Folders, select All Tasks, and select Configure Shadow Copies.

• Select the volume where you want to make changes, and then select Settings.

• In the Settings dialog box, change the Maximum Size setting to either of the following: No Limit or a size large enough to suit the requirements of your installation and your usage of VSS.

• If the backup fails with status code 156, consult the bpfis legacy log, in /usr/openv/netbackup/logs (UNIX) or install_path\NetBackup\logs (Windows). If the bpfis directory does not already exist, you must create it and rerun the job.

  If necessary, increase the logging level and retry the job.

  See "About logs" in the NetBackup Logging Reference Guide.

• On Windows clients, when restoring files from a backup that is made with the NAS_Snapshot method, log into the NetBackup Client Service as the Administrator account, not as the local system account. Otherwise, the backup fails with status 156.

  • In Windows Services, double-click the NetBackup Client Service.

  • Then check the Log On tab: if the service is not logged on as Administrator, stop the service.

  • Change the logon to the Administrator account and restart the service.

  • Retry the restore.

• The file system that is specified as a snapshot source is not mounted. In this case, you may see the following in the /usr/openv/netbackup/logs/bpfis log:

  17:12:51 bpfis: FTL - snapshot creation failed, status 156
  17:12:51 bpfis: INF - EXIT STATUS 156: snapshot error 
  encountered

  And the following in the /usr/openv/netbackup/logs/bpfis log:

  17:12:51 onlfi_vfms_logf: INF - cannot snap_on, err: 5
  17:12:51 delete_mount_point: INF - Deleted mount point 
  /tmp/__jody_test:20958
  17:12:51 onlfi_freeze: FTL - VfMS error 11; see following 
  messages:
  17:12:51 onlfi_freeze: FTL - Fatal method error
  17:12:51 onlfi_freeze: FTL - vfm_freeze: method: nbu_snap, type: 
  FIM, function: nbu_snap_freeze
  17:12:51 onlfi_freeze: FTL - VfMS method error 5; see following 
  message:
  17:12:51 onlfi_freeze: FTL - nbu_snap_freeze: Cannot turn on 
  snapshot; snapshot source=/opt, cache=/dev/rdsk/c1t3d1s0, snap 
  error=5
  17:12:51 onlfi_thaw: WRN - / is not frozen

  Make sure that the file system that is specified for the snapshot source has been mounted.

• The file system that is specified as the snapshot source does not correspond to the file system that contains the actual files (as opposed to symbolic links to the files). The mounted file system for the snapshot source must contain the actual files, not symbolic links. If items in the file list, such as /oracle, is a symbolic link to /export/home/oracle, the snapshot source must specify /export, or /export/home, not /oracle.

• VxVM is selected as the snapshot method but the snapshot source is not configured over a Veritas Volume Manager (VxVM) volume. In this case, you may see the following in the /usr/openv/netbackup/logs/bpfis log:

  17:12:51 bpfis: FTL - snapshot creation failed, status 156
  17:12:51 bpfis: INF - EXIT STATUS 156: snapshot error 
  encountered

  And something like the following in the /usr/openv/netbackup/logs/bpfis log:

  17:12:51 onlfi_vfms_logf: INF - vxvm_freeze: Snapshot source 
  /cockpit1 on device /dev/dsk/c1t0d0s6 is not on a VxVM volume
  17:12:51 delete_mount_point:   INF - Deleted mount point 
  /tmp/_cockpit1_coc_group1:3518
  17:12:51 onlfi_freeze: FTL - VfMS error 11; see following 
  messages:
  17:12:51 onlfi_freeze: FTL - Fatal method error
  17:12:51 onlfi_freeze: FTL - vfm_freeze: method: vxvm, type: 
  FIM, function: vxvm_freeze
  17:12:51 onlfi_freeze: FTL - VfMS method error 9; see following 
  message:
  17:12:51 onlfi_freeze: FTL - vxvm_freeze: Snapshot source 
  /cockpit1 on device /dev/dsk/c1t0d0s6 is not on a VxVM volume
  17:12:51 onlfi_thaw: INF - fim=vxvm
  17:12:51 onlfi_thaw: WRN - /cockpit1 is not frozen

  Make sure that the snapshot source is configured over a Veritas Volume Manager (VxVM) volume.

• VxVM was selected as the snapshot method, but a Veritas Volume Manager snapshot mirror of the snapshot source volume had not been created before you ran the backup, or another backup is currently running that uses the snapshot mirror. In either case, you may see the following in the /usr/openv/netbackup/logs/bpfis log:

  17:12:51 onlfi_freeze: FTL - VfMS error 11; see following 
  messages:
  17:12:51 onlfi_freeze: FTL - Fatal method error
  17:12:51 onlfi_freeze: FTL - vfm_freeze: method: vxvm, type: 
  FIM, function: vxvm_freeze
  17:12:51 onlfi_freeze: FTL - VfMS method error 3; see following 
  message:
  17:12:51 onlfi_freeze: FTL - find_ready_snapshot: Cannot find 
  available snapshot mirror

  See the NetBackup Snapshot Client Administrator's Guide for information on how to create a snapshot mirror on the client before you run the backup.

• VxVM was selected as the snapshot method, and a Veritas Volume Manager snapshot mirror of the snapshot source volume has been created. However, two different backup jobs (A and B) try to back up the same volume (for example, vol01), but job A starts before job B. After an available snapshot mirror is found, a brief pause occurs before the snapshot is formed. Job B that runs slightly behind job A may try to create a snapshot of the snapshot mirror immediately before job A creates the snapshot and gets the lock on it.

  In this case, you may see the following in the /usr/openv/netbackup/logs/bpfis log:

  17:12:51 onlfi_freeze: FTL - VfMS error 11; see following 
  messages:
  17:12:51 onlfi_freeze: FTL - Fatal method error
  17:12:51 onlfi_freeze: FTL - vfm_freeze: method: vxvm, type: 
  FIM, function: vxvm_freeze
  17:12:51 onlfi_freeze: FTL - VfMS method error 3; see following 
  message:
  17:12:51 onlfi_freeze: FTL - vxvm_freeze: Command failed with 
  status=11: /usr/sbin/vxassist -g rootdg snapshot vol01 
  VfMSCAAu7a4Uw </dev/null>/var/tmp/VfMSAAAs7a4Uw 
  2>/var/tmp/VfMSBAAt7a4Uw

  The job that was unable to get a lock (job B in the preceding example) fails, and must be run again.

• When you use nbu_snap as a snapshot method, you may have stale snapshots if status code 156 occurs with the following messages in the /usr/openv/netbackup/logs/bpfis log. (Stale snapshots are those that nbu_snap did not automatically delete.)

  17:12:51 onlfi_freeze: FTL - VfMS error 11; see following 
  messages:
  17:12:51 onlfi_freeze: FTL - Fatal method error
  17:12:51 onlfi_freeze: FTL - vfm_freeze: method: nbu_snap, type: 
  FIM, function: nbu_snap_freeze
  17:12:51 onlfi_freeze: FTL - VfMS method error 5; see following 
  
  message:
  17:12:51 onlfi_freeze: FTL - nbu_snap_freeze: Cannot turn on 
  snapshot; snapshot source=/oracle/ufs_r, 
  cache=/dev/rdsk/c4t1d11s4,snap error=11

• Look for stale snapshots by running the /usr/openv/netbackup/bin/driver/snaplist command when there are no active backups running. If the snaplist command shows cache entries, there are stale snapshots. Nothing is displayed if there are no stale snapshots.

  Example snaplist output:

  id  ident        size     cached    minblk     err time
  43  6515    8390970    0          0          0  11/16/00 13:31:36
  device = /dev/rdsk/c1t6d0s0
  cache  = /dev/rdsk/c1t6d0s7

• Use the snapoff command to remove the stale snapshot, as follows:

  /usr/openv/netbackup/bin/driver/snapoff id

  Where id is the ID from the snaplist output (such as 43 in the preceding example).

• If a backup using the VxFS_Checkpoint snapshot method failed, the NetBackup bpbkar process should automatically remove the clone. Sometimes, however, bpbkar is unable to remove the clone. In this case, you may see messages such as the following in the /usr/openv/netbackup/logs/bpfis log:

  15:21:45.716 [4236] <4> create_mount_point: INF - Created mount 
  point /tmp/_vtrax_test:4236
  15:21:45.869 [4236] <2> onlfi_vfms_logf: INF - vxfs clone handle 
  : 9600344
  15:21:45.870 [4236] <2> onlfi_vfms_logf: INF - 
  VxFS_Checkpoint_freeze: Cannot create checkpoint; status=17
  15:21:45.872 [4236] <4> delete_mount_point: INF - Deleted mount 
  point /tmp/_vtrax_test:4236
  15:21:45.873 [4236] <32> onlfi_freeze: FTL - VfMS error 11; see 
  following messages:
  15:21:45.873 [4236] <32> onlfi_freeze: FTL - Fatal method error 
  was reported
  15:21:45.873 [4236] <32> onlfi_freeze: FTL - vfm_freeze: method: 
  VxFS_Checkpoint, type: FIM, function: VxFS_Checkpoint_freeze
  15:21:45.873 [4236] <32> onlfi_freeze: FTL - VfMS method error 
  17; see following message:
  15:21:45.874 [4236] <32> onlfi_freeze: FTL - 
  VxFS_Checkpoint_freeze: Cannot create checkpoint; status=17

  Remove the clone as follows.

  Note:

  If the checkpoint is not removed, you cannot use VxFS_Checkpoint to back up any data in the file system where the checkpoint is mounted.

• List the name of the checkpoint by entering the following VxFS command:

  /usr/lib/fs/vxfs/fsckptadm list /file_system

  Where file_system is the name of the file system where the checkpoint is mounted. A sample output follows. In this example, /vtrax_test is the file system and fi_ckpt is the name of the checkpoint.

  /vtrax_test
  fi_ckpt:
  ctime = Mon Nov 12 10:08:13 2001
  mtime = Mon Nov 12 10:08:13 2001
  flags = largefiles

• Remove the checkpoint by entering the following:

  /usr/lib/fs/vxfs/fsckptadm remove checkpoint /file_system

• If the checkpoint cannot be removed, unmount the checkpoint and retry the first step in this procedure.

• If a snapshot backup fails using TimeFinder, ShadowImage, or BusinessCopy method, there may be a VxVM clone left over from a previous backup. You may see messages similar to the following in the /usr/openv/netbackup/logs/bpfis log:

  19:13:07.686 [14981] <2> onlfi_vfms_logf: INF - do_cmd: Command 
  failed with status=20: /usr/openv/netbackup/bin/bpdgclone -g 
  wil_test -n vol01 -f /var/tmp/HDSTFCAAs7aOqD </dev/null 
  >/var/tmp/VfMSAAAq7aOqD 2>/var/tmp/VfMSBAAr7aOqD
  19:13:07.687 [14981] <2> onlfi_vfms_logf: INF - --- Dumping file 
  /var/tmp/VfMSAAAq7aOqD (stdout):
  19:13:07.687 [14981] <2> onlfi_vfms_logf: INF - --- End of file 
  /var/tmp/VfMSAAAq7aOqD
  19:13:07.687 [14981] <2> onlfi_vfms_logf: INF - --- Dumping file 
  /var/tmp/VfMSBAAr7aOqD (stderr):
  19:13:07.687 [14981] <2> onlfi_vfms_logf: INF -     clone group 
  and volume already exists
  19:13:07.688 [14981] <2> onlfi_vfms_logf: INF - --- End of file 
  /var/tmp/VfMSBAAr7aOqD

  NetBackup automatically creates VxVM clones for TimeFinder, ShadowImage, or BusinessCopy backups of the data that is configured over volumes. After the backup has completed, NetBackup removes the VxVM clone. In this case, a system crash or restart may have prevented the removal. Remove the clone as follows.

  (Do the following on the client or alternate client, depending on the type of backup.)

• When no backups are running, use the following VxVM command to list any clones: vxdg list

  The clone name is of the form clone_disk_group_clone.

• To remove the clone, enter the following:

  /usr/openv/netbackup/bin/bpdgclone -g disk_group -n volume -c

  For example:

  /usr/openv/netbackup/bin/bpdgclone -g wil_test -n vol01 -c

  Where wil_test is the name of the disk group and volo1 is the name of the VxVM volume.

  For more information on how to remove a VxVM clone, see the NetBackup Snapshot Client Administrator's Guide. For vxdg, see the Veritas Volume Manager Administrator's Guide.

• Before running the backup again, resynchronize the primary disk with the secondary disk. For assistance, see the NetBackup Snapshot Client Administrator's Guide.

• If a snapshot backup fails using the FlashSnap or VVR snapshot method, a VxVM snapshot may be left over from a previous backup. You may see messages similar to the following in the /usr/openv/netbackup/logs/bpfis log:

  14:41:15.345 [22493] <32> onlfi_freeze: FTL - VfMS error 11; see 
  following messages:
  14:41:15.345 [22493] <32> onlfi_freeze: FTL - Fatal method error 
  was reported
  14:41:15.345 [22493] <32> onlfi_freeze: FTL - vfm_freeze_commit: 
  method: FlashSnap, type: FIM, function: FlashSnap_freeze_commit
  14:41:15.345 [22493] <32> onlfi_freeze: FTL - VfMS method error 
  8; see following message:
  14:41:15.345 [22493] <32> onlfi_freeze: FTL - 
  vxvm__find_ready_snapshot: Cannot find available snapshot mirror

  NetBackup automatically creates VxVM snapshots for backups of the data that is configured over volumes. After the backup completes, NetBackup removes the VxVM snapshot. In this case, a system crash or restart may have prevented the removal. Remove the snapshot as follows.

  For FlashSnap:

  (Do the following on the client or alternate client, depending on the type of backup.)

• Find the VxVM disk group:

        vxdg list

  The format of the disk group name is as follows:

        primaryhost_diskgroup_split

  If vxdg list does not show the disk group, the group might have been deported. You can discover all the disk groups including deported ones by entering:

        vxdisk -o alldgs list

  The disk groups that are listed in parentheses are not imported on the local system.

• Deport the VxVM disk group:

        vxdg deport primaryhost_diskgroup_split

  Enter the following on the primary (original) client:

• Import and join the VxVM disk group:

        vxdg import primaryhost_diskgroup_split
        vxrecover -g primaryhost_diskgroup_split -m
        vxdg join primaryhost_diskgroup_split diskgroup

• Start the volume and snap back the snapshot volume:

      vxvol -g primaryhost_diskgroup_split start 
  SNAP_diskgroup_volume
      vxassist snapback SNAP_diskgroup_volume

  For VVR, on the alternate client:

• Enter the following to display unsynchronized mirror disks:

      vxprint -g diskgroup

• Enter the following to resynchronize the mirror disks:

        vxassist -g diskgroup -v volume snapback

• When you use a snapshot method such as VxFS_Checkpoint to back up a Veritas File System (VxFS), the backup fails if the VxFS license has expired. Messages such as the following appear in the /usr/openv/netbackup/logs/bpfis log:

  11:37:42.279 [24194] <2> onlfi_vfms_logf: INF - 
  VxFS_Checkpoint_freeze: Cannot open checkpoint; status=100
  11:37:42.283 [24194] <4> delete_mount_point: INF - Deleted mount 
  point /tmp/_vrts_frzn_img__test1_24194
  11:37:42.283 [24194] <32> onlfi_freeze_fim_fs: FTL - VfMS error 
  11; see following messages:
  11:37:42.283 [24194] <32> onlfi_freeze_fim_fs: FTL - Fatal 
  method error was reported
  11:37:42.284 [24194] <32> onlfi_freeze_fim_fs: FTL - vfm_freeze: 
  method: VxFS_Checkpoint, type: FIM, function: 
  VxFS_Checkpoint_freeze
  11:37:42.284 [24194] <32> onlfi_freeze_fim_fs: FTL - VfMS method 
  error 100; see following message:
  11:37:42.284 [24194] <32> onlfi_freeze_fim_fs: FTL - 
  VxFS_Checkpoint_freeze: Cannot open checkpoint; status=100

  Obtain a new VxFS license and retry the backup.

• If the backup is enabled for instant recovery with either the VxVM or VVR snapshot method, your VxVM mirrors may not be properly configured. In this case, you may see the following in the /usr/openv/netbackup/logs/bppfi log on the client (when verbose mode is set high).

  13:43:39.095 [16375] <2> onlfi_vfms_logf: INF - Executing 
  command:
  13:43:39.095 [16375] <2> onlfi_vfms_logf: INF - 
  /usr/sbin/vxprint -g rootdg -q -t -e 'assoc="pfi_concat"' 
  </dev/null >/var/tmp/VfMSAA Arja4.F 2>/var/tmp/VfMSBAAsja4.F
  13:43:39.215 [16375] <2> onlfi_vfms_logf: INF - 
  pfi_find_snapdone: 0 SNAPDONE plexes found
  
  13:43:39.215 [16375] <2> onlfi_vfms_logf: INF - Executing 
  command:
  13:43:39.215 [16375] <2> onlfi_vfms_logf: INF - 
  /usr/sbin/vxassist -g rootdg snapprint pfi_concat  </dev/null 
  >/var/tmp/VfMSAAArja4.F 2>/var/tmp/VfMSBAAsja4.F
  13:43:39.512 [16375] <2> onlfi_vfms_logf: INF - 0 active plexes 
  for /rootdg/pfi_concat:  0 are PFI  0 non-PFI
  13:43:39.512 [16375] <2> onlfi_vfms_logf: INF - 
  pfi_find_active.3309: exiting with VXVM_E_SYS = 3
  13:43:39.512 [16375] <2> onlfi_vfms_logf: INF - 
  pfi_snapshot.3866: No PFI snapshot. err= 3

  Configure the VxVM mirrors as described in the Instant Recovery chapter of the NetBackup Snapshot Client Administrator's Guide.

• When you use the VxFS_Checkpoint snapshot method, the backup fails if the client's file system does not support mountable checkpoints using the Storage Checkpoint feature. Messages such as the following appear in the /usr/openv/netbackup/logs/bpfis log:

  14:54:27.530 [23563] <32> onlfi_freeze_fim_fs: FTL - VfMS error 
  11; see following messages:
  14:54:27.530 [23563] <32> onlfi_freeze_fim_fs: FTL - Fatal 
  method error was reported
  14:54:27.530 [23563] <32> onlfi_freeze_fim_fs: FTL - vfm_freeze: 
  method: VxFS_Checkpoint, type: FIM, function: 
  VxFS_Checkpoint_freeze 14:54:27.531 [23563] <32> 
  onlfi_freeze_fim_fs: FTL - VfMS method error 2; see following 
  message:
  14:54:27.531 [23563] <32> onlfi_freeze_fim_fs: FTL - open_ckpt: 
  Cannot open checkpoint on /mnt_vxvm/2G_concat : 
  fsckpt_get_api_version returns 1; mountable checkpoints not 
  supported with this version

  Do one of the following:

  • Upgrade the client file system to a version that supports mountable VxFS Storage Checkpoints.

  • Configure the policy with a snapshot method that supports the client's current file system.