How to troubleshoot and correct the error "Fatal error: disk write failure /usr/openv/db/data/NBDB.log"

Description

Administrators may encounter the following error in the /usr/openv/db/log/server.log file for a few common reasons:

Fatal error: disk write failure /usr/openv/db/data/NBDB.log

The most common reason for this error is due to the filesystem that houses the database being to small.

Running the 'df -k' command will show the space available on the file system. If the file system space is adequate, another reason this can occur is due to the ulimit setting for the maximum file size being set too small.

Example:

$ ulimit-a
time(seconds)unlimited
file(blocks)2097151
data(kbytes)131072
stack(kbytes)32768
memory(kbytes)32768
coredump(blocks)2097151
nofiles(descriptors)8000
 

The file(blocks) setting above is 2,097,151 in this example. Multiplying by the block size (512 bytes) yields the maximum byte count to which a file can grow as limited by the operating system.

In this case:
2097151 * 512 =1073741312 bytes

With these settings in the example, the NBDB.log cannot exceed 1,073,741,312 bytes.

Reviewing the size of the files using ls would show whether or not the size was at the maximum:

-rw------- 1 root system 5009408 Oct 0406:11 BMRDB.db
-rw-rw-rw- 1 root system 327680 Oct 04 06:11 BMRDB.log
-rw------- 1 root system 28389376Oct 04 03:11 BMR_DATA.db
-rw------- 1 root system 26226688 Aug 27 06:00 BMR_INDEX.db
-rw------- 1 root system 75157504Oct 04 06:21 EMM_DATA.db
-rw------- 1 root system 26226688 Oct 04 06:20 EMM_INDEX.db
-rw------- 1 root system 6778880 Oct04 06:21 NBDB.db
-rw-rw-rw- 1 root system 1073741312 Oct 04 06:21 NBDB.log   <== file at max size.

In this case the application will not run until the ulimit is increased for the file(blocks) size setting.

Resolution

Use the ulimit -f or other appropriate procedure for the operating system, on which NetBackup, is running to increase the file(blocks) size.

Note: For additional details, see the Related Article: Minimum O/S ulimit settings on primary and media server Linux/UNIX platforms.