提交yarn任务报错空指针org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster

Love The Way You Lie 2024-03-26 17:12 20阅读 0赞

新建的hadoop集群提交yarn任务后报错，hive任务也执行不了。  
报错如下：

ERROR [Listener at 0.0.0.0/45871] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
    org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NullPointerException
        at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:178)
        at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:122)
        at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAllocator.java:280)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStart(MRAppMaster.java:979)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
        at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1293)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$6.run(MRAppMaster.java:1761)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1757)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1691)
    Caused by: java.lang.NullPointerException
        at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.getHttpPort(MRClientService.java:177)
        at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:159)
        ... 14 more

看了下这个空指针MRClientService.getHttpPort(MRClientService.java:177)  
@Override  
public int getHttpPort() \{  
return webApp.port();  
\}  
webApp的port没有获取到, 再看port()方法就是获取httpServer地址的端口  
public int port() \{  
InetSocketAddress addr = checkNotNull(httpServer, “httpServer”)  
.getConnectorAddress(0);  
return addr == null ? -1 : addr.getPort();  
\}

看网上说是yarn.resourcemanager.webapp.address没配置导致没能获取到port。  
在yarn-site.xml中配置了yarn.resourcemanager.webapp.address之后就正常了。

如果是HA机制下yarn-site.xml需要加入以下配置:

<property>
    	<name>yarn.resourcemanager.hostname.rm1</name>
    	<value>xxx1</value>
    </property>
    
    <property>
    <name>yarn.resourcemanager.webapp.address.rm1</name>
    <value>xxx1:8088</value>
    </property>
    
    
    <property>
    	<name>yarn.resourcemanager.hostname.rm2</name>
    	<value>hadoop103</value>
    </property>
    <property>
    <name>yarn.resourcemanager.webapp.address.rm2</name>
    <value>xxx2:8088</value>
    </property>

可以参考这个配置[https://blog.csdn.net/m0\_51111980/article/details/126085577][https_blog.csdn.net_m0_51111980_article_details_126085577]  
xxx1主机名的值从yarn.resourcemanager.hostname中找,如果是HA那么是类似这样的

参考:  
\[1\] https://blog.csdn.net/weixin\_44812855/article/details/112716670  
\[2\] https://blog.csdn.net/m0\_51111980/article/details/126085577  
\[3\] https://www.cnblogs.com/dreamboy/archive/2018/07/10/9289631.html  
\[4\] https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/ResourceManagerHA.html

[https_blog.csdn.net_m0_51111980_article_details_126085577]: https://blog.csdn.net/m0_51111980/article/details/126085577