Monday, February 24, 2014

Apache Hadoop - How to Install a Three Nodes Cluster (3)

Previous tutorial:
Apache Hadoop - How to Install a Three Nodes Cluster (1)
Apache Hadoop - How to Install a Three Nodes Cluster (2)
Apache Hadoop - How to Install a Three Nodes Cluster (4)

Configure Secondary Namenode:
To configure secondary namenode is easy, in our tutorial, we choose hadoop2 server to be the secondary namenode. So on hadoop2, open hdfs-site.xml file:

# vi hdfs-site.xml

Add property:
        <description> A comma separated list of paths. Use the list of directories from $FS_CHECKPOINT_DIR.
                For example, /grid/hadoop/hdfs/snn,sbr/grid1/hadoop/hdfs/snn,sbr/grid2/hadoop/hdfs/snn

Then save hdfs-site.xml file,run the following command:
$HADOOP_PREFIX/sbin/ --config $HADOOP_CONF_DIR --script hdfs start secondarynamenode

Open your web browser, go to "http://hadoop2:50090", you can check the status of Secondary Namenode.

1. Stack guard warning 1:
If you see warnings like this:
Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /opt/hadoop-2.2.0/lib/native/ which might have disabled stack guard. The VM will try to fix the stack guard now.

It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.

Run the following command to fix the warning:
# execstack -c /opt/hadoop-2.2.0/lib/native/

2. URI path warning:
14/02/19 11:22:23 WARN common.Util: Path /data/nn1 should be specified as a URI in configuration files. Please update hdfs configuration.

This is a warning that your "" must be "file:///data/nn1" and not simple "/data/nn1". That is, it must now be a proper URI. It is harmless in its present form though, but I recommend changing it to avoid incompatibility in future when direct path

support may get yanked or cause wrong FS assumptions.

To get rid of this warning, change:

3. JobHistoryServer can’t create done directory:
13/12/31 11:21:10 INFO hs.JobHistoryServer: registered UNIX signal handlers for [TERM, HUP, INT]
13/12/31 11:21:12 INFO hs.JobHistory: JobHistory Init
13/12/31 11:21:12 ERROR security.UserGroupInformation: PriviledgedActionException as:mapred (auth:SIMPLE) cause:org.apache.hadoop.fs.UnsupportedFileSystemException: No AbstractFileSystem for scheme:
13/12/31 11:21:12 INFO service.AbstractService: Service org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager failed in state INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error creating done directory: [null]
at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.serviceInit(

It is possible that in your core-site.xml file, your fs.defaultFS value was only set to “NAMENODE” instead of “hdfs://NAMENODE:PORT”. Try to prepend the “hdfs://” and adding “:8020″ as the port at the end. Also you should check directories permissions in your HDFS. Make sure you are running JobHistoryServer as $MAPRED_USER (in my case is mapred).

Assuming you are using /mr-history/tmp as “mapreduce.jobhistory.intermediate-done-dir” and /mr-history/done as “mapreduce.jobhistory.done-dir”.

$ hadoop fs -mkdir -p /mr-history/tmp
$ hadoop fs -chmod -R 1777 /mr-history/tmp
$ hadoop fs -mkdir -p /mr-history/done
$ hadoop fs -chmod -R 1777 /mr-history/done
$ hadoop fs -chown -R $MAPRED_USER:$HDFS_USER /mr-history

No comments: