解决 hadoop 2.x 配置 yarn 运行任务 Running job 卡住

骑猪看日落 2022-10-10 01:27 244阅读 0赞

错误信息：  
21/06/24 18:56:40 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using  
builtin-java classes where applicable  
21/06/24 18:56:41 INFO client.RMProxy: Connecting to ResourceManager at /192.168.100.20:8032  
21/06/24 18:56:42 INFO input.FileInputFormat: Total input files to process : 1  
21/06/24 18:56:42 INFO mapreduce.JobSubmitter: number of splits:1  
21/06/24 18:56:43 INFO mapreduce.JobSubmitter: Submitting tokens for job: job\_1624586163461\_0001  
21/06/24 18:56:43 INFO conf.Configuration: resource-types.xml not found  
21/06/24 18:56:43 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.  
21/06/24 18:56:43 INFO resource.ResourceUtils: Adding resource type - name = memory-mb, units = Mi, type =  
COUNTABLE  
21/06/24 18:56:43 INFO resource.ResourceUtils: Adding resource type - name = vcores, units = , type = COUNTABLE  
21/06/24 18:56:43 INFO impl.YarnClientImpl: Submitted application application\_1624586163461\_0001  
21/06/24 18:56:43 INFO mapreduce.Job: The url to track the job:  
http://192.168.100.20:8088/proxy/application\_1624586163461\_0001/  
21/06/24 18:56:43 INFO mapreduce.Job: Running job: job\_1624586163461\_0001

分析原因：因为yarn-site.xml配置文件中yarn.resourcemanager.hostname属性配置错误

解决方法一：如果yarn.resourcemanager.hostname属性值写成IP地址，仍然卡住，建议使用hostname  
\# 修改hostname  
vi /etc/hosts

192.168.100.10    hadoop10

\# 重启使hosts生效  
reboot

\# 修改yarn-site.xml配置文件  
vi yarn-site.xml

<property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    
    <property>
        <name>yarn.resourcemanager.hostname</name>
        <value>hadoop10</value>
    </property>

解决方法二：不使用yarn，删除mapred-site.xml中的yarn配置，不推荐，治标不治本  
vi mapred-site.xml