执行wordcount例子报异常-蒲公英云

执行wordcount例子报异常

异常一：
2018-01-09 02:50:43,320 INFO mapreduce.Job: Job job_1515466125577_0002 failed with state FAILED due to: Application application_1515466125577_0002 failed 2 times due to AM Container for appattempt_1515466125577_0002_000002 exited with exitCode: 127
Failing this attempt.Diagnostics: [2018-01-09 02:50:42.706]Exception from container-launch.
Container id: container_1515466125577_0002_02_000001
Exit code: 127

[2018-01-09 02:50:42.750]Container exited with a non-zero exit code 127. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
/bin/bash: /bin/java: No such file or directory

解决方法：在中添加adoop-3.0.0/libexec/hadoop-config.sh文件中添加JAVA_HOME环境变量

异常二：
2018-01-09 03:25:37,250 INFO mapreduce.Job: Job job_1515468264727_0001 failed with state FAILED due to: Application application_1515468264727_0001 failed 2 times due to AM Container for appattempt_1515468264727_0001_000002 exited with exitCode: 1
Failing this attempt.Diagnostics: [2018-01-09 03:25:36.447]Exception from container-launch.
Container id: container_1515468264727_0001_02_000001
Exit code: 1

[2018-01-09 03:25:36.499]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster

Please check whether your etc/hadoop/mapred-site.xml contains the below configuration:

yarn.app.mapreduce.am.env
HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}

mapreduce.map.env
HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}

mapreduce.reduce.env
HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}

解决方法：按错误提示，在mapred-site.xml配置文件中添加hadoop根目录

异常三：
2018-01-09 03:55:18,863 INFO mapreduce.Job: Job job_1515470038765_0001 failed with state FAILED due to: Application application_1515470038765_0001 failed 2 times due to AM Container for appattempt_1515470038765_0001_000002 exited with exitCode: -103
Failing this attempt.Diagnostics: [2018-01-09 03:55:17.480]Container [pid=728,containerID=container_1515470038765_0001_02_000001] is running beyond virtual memory limits. Current usage: 146.5 MB of 1 GB physical memory used; 2.7 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1515470038765_0001_02_000001 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 741 728 728 728 (java) 634 19 2852925440 37258 /usr/local/jdk1.8.0_131/bin/java -Djava.io.tmpdir=/home/hadoop/tmp/nm-local-dir/usercache/hadoop/appcache/application_1515470038765_0001/container_1515470038765_0001_02_000001/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/home/hadoop/hadoop-3.0.0/logs/userlogs/application_1515470038765_0001/container_1515470038765_0001_02_000001 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster
解决方法：在yarn-site.xml中添加以下配置

yarn.nodemanager.vmem-pmem-ratio
5
Ratio between virtual memory to physical memory when setting memory limits
for containers. Container allocations are expressed in terms of physical memory, and
virtual memory usage is allowed to exceed this allocation by this ratio.