Search <book_title>...

NetBackup™ for Hadoop Administrator's Guide

Last Published: 2022-09-08

Product(s): NetBackup (10.1)

Configuring NetBackup for a highly-available Hadoop cluster

To protect a highly-available Hadoop cluster, when you configure NetBackup for Hadoop cluster:

Specify one of the NameNodes (primary) as the client in the BigData policy.
Specify the same NameNode (primary and fail-over) as application server when you execute the tpconfig command.
Create a hadoop.conf file, update it with the details of the NameNodes (primary and fail-over), and copy it to all the backup hosts. The hadoop.conf file is in JSON format.
Hostname and port of the NameNode must be same as you have specified with the http address parameter in the core-site.xml of the Hadoop cluster.
User name of the primary and fail-over NameNode must be same.
Do not provide a blank value for any of the parameters, or the backup job fails.

To update the hadoop.conf file for highly-available Hadoop cluster

Update the hadoop.conf file with the following parameters:

{
 "application_servers": 
 {
		"hostname_of_primary_namenode1":
		{
			"failover_namenodes":
			[
				{
					"hostname": "hostname_of_failover_namenode1",
					"port": port_of_failover_namenode1
  		}				
	 	],
		 "port":port_of_primary_namenode1
  }
 }
}

If you have multiple Hadoop clusters, use the same hadoop.conf file to update the details. For example,

{
	"application_servers": 
 {
		"hostname_of_primary_namenode1":
  {
		 "failover_namenodes":
   [
			 {
				 "hostname": "hostname_of_failover_namenode1",
				 "port": port_of_failover_namenode1
			 }
	 	],
   "port"::port_of_primary_namenode1
  },
  "hostname_of_primary_namenode2":
  {
		 "failover_namenodes":
   [
			 {
			  "hostname": "hostname_of_failover_namenode2",
			  "port": port_of_failover_namenode2
			 }
	 	],
			"port":port_of_primary_namenode2
	 }
 }
}

Copy this file to the following location on all the backup hosts:
/usr/openv/var/global/