A quick way to test your map and reduce task numbers for you cluster, good for cluster tuning.
You can specify map or reduce tasks from command line, and this example jar file is built in:
$ hadoop jar /opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar grep input output 'dfs[a-z.]+' $ hadoop jar /opt/cloudera/parcels/CDH-4.2.1-1.cdh4.2.1.p0.5/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar grep -D mapred.map.tasks=8 -D mapred.reduce.tasks=6 input output 'dfs[a-z.]+'
For other properies:
http://archive.cloudera.com/cdh4/cdh/4/mr1/mapred-default.html
2 comments:
hi ,you have gathered a valuable information on Hadoop...., and i am much impressed with the information and it is useful for Hadoop Learners.These blogs are valuable because these are providing such informative information for all the people.
Hadoop Training in hyderabad
Thanks Sundara for your comment.
Post a Comment