Hadoop启动datanode结点All specified directories are failed to load报错解决方案

今天药忘吃喽~ 2022-01-30 03:19 464阅读 0赞

# 前言 #

因重启阿里云服务器，需要重新启动格式化namenode，重新启动结点后发现，datanode没有启动成功，反复几次后还是不行，最后查看日志文件发现报**All specified directories are failed to load**错误。在网上查了好一会，原因在于每次格式化时，namenode的clusterID会改变，而datanode的clusterID不会改变，造成namenode和datanode的clusterID不一致。以下是解决过程

# 解决过程 #

2019-05-16 17:57:28,144 WARN org.apache.hadoop.hdfs.server.common.Storage: java.io.IOException: Incompatible clusterIDs in /tmp/hadoop-root/dfs/data: namenode clusterID = CID-6dbccc7d-1bc2-43e4-a564-f72664b2d532; datanode clusterID = CID-5d721c74-9d56-4504-a08b-0d2e10b925c3
    2019-05-16 17:57:28,144 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid unassigned) service to phadoop01/172.18.77.199:9000. Exiting. 
    java.io.IOException: All specified directories are failed to load.
            at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:478)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1342)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1308)
            at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:314)
            at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:226)
            at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:867)
            at java.lang.Thread.run(Thread.java:748)
    2019-05-16 17:57:28,146 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool <registering> (Datanode Uuid unassigned) service to phadoop01/172.18.77.199:9000
    2019-05-16 17:57:28,247 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Removed Block pool <registering> (Datanode Uuid unassigned)
    2019-05-16 17:57:30,251 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Exiting Datanode
    2019-05-16 17:57:30,252 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 0
    2019-05-16 17:57:30,301 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG: 
    /************************************************************

以上为日志文件报错

此问题有两种解决方法

**一、更改datanode的clusterID，使其与namenode的ID一致**  
打开namenode的VERSION文件

vim /tmp/hadoop-root/dfs/name/current/VERSION

复制namenode的clusterID  
![namenodeID][]  
打开datanode的VERSION文件，把clusterID修改为namenode的clusterID就好了

vim /tmp/hadoop-root/dfs/data/current/VERSION

**二、删除data目录的文件**

cd到tmp目录下的data目录将data里面的文件全删了

cd /tmp/hadoop-root/dfs/data

![在这里插入图片描述][2019051720271048.png]![在这里插入图片描述][20190517202745888.png]

最后，无论是使用哪种方法，重启datanode结点，为了省事我用了启动所有结点的命令

start-all.sh

![在这里插入图片描述][20190517203243906.png]  
datanode就启动成功了

# 总结 #

这算是比较常见的问题，只要在启动结点时后，未关闭就再格式化就会出现这个问题，有日志报错还是很好解决的。

[namenodeID]: /images/20220130/75e18bf71d104844951915bb779648da.png
[2019051720271048.png]: https://img-blog.csdnimg.cn/2019051720271048.png
[20190517202745888.png]: /images/20220130/33cfdbbc66a247c6a44bbbdd44787d6f.png
[20190517203243906.png]: /images/20220130/16035c3fa4f64b8692f1b12e4e63a6a6.png