如何彻底删除Kafka中的topic (marked for deletion)

我会带着你远行 2022-05-10 11:24 1111阅读 0赞

工作中因为各种原因， 例如topic中消息堆积的太多，或者kafka所在磁盘空间满了等等，可能需要彻底清理一下kafka topic，那么如何彻底删除topic？  
**方法一（配置delete.topic.enable=true）**  
  修改kafaka配置文件server.properties， 添加delete.topic.enable=true，重启kafka，之后通过kafka命令行就可以直接删除topic  
  通过命令行删除topic：

./bin/kafka-topics.sh --delete --zookeeper \{zookeeper server\} --topic \{topic name\}

**方法二（没有配置delete.topic.enable=true）**

1、通过命令行删除topic：

./bin/kafka-topics.sh --delete --zookeeper \{zookeeper server\} --topic \{topic name\}

因为kafaka配置文件中server.properties没有配置delete.topic.enable=true，此时的删除并不是真正的删除，只是把topic标记为：marked for deletion  
  你可以通过命令：./bin/kafka-topics --zookeeper \{zookeeper server\} --list 来查看所有topic  
2、删除kafka存储目录（server.properties文件log.dirs配置，默认为"/tmp/kafka-logs"）相关topic目录  
![在这里插入图片描述][70]

3， 若想真正删除它，需要登录zookeeper客户端：

命令：./bin/zkCli.sh

找到topic所在的目录：ls /brokers/topics

执行命令：rmr /brokers/topics/\{topic name\}即可，此时topic被彻底删除。

另外被标记为marked for deletion的topic你可以在zookeeper客户端中通过命令获得：ls /admin/delete\_topics/\{topic name\}，如果你删除了此处的topic，那么marked for deletion 标记消失

zookeeper 的config中也有有关topic的信息： ls /config/topics/\{topic name\}暂时不知道有什么用

**总结**

彻底删除topic：  
1、确保kafka的配置文件server.proeprties中设置delete.topic.enable=true，如果没有， 确保cluster的所有kafka配置文件设置该参数并重启，然后直接通过命令删除，如果命令删除不了，直接通过zookeeper命令行删除掉broker下的topic。

2、删除kafka存储目录（server.properties文件log.dirs配置，默认为"/tmp/kafka-logs"）相关topic目录

参考文档：  
1， [https://stackoverflow.com/questions/17730905/is-there-a-way-to-delete-all-the-data-from-a-topic-or-delete-the-topic-before-ev][https_stackoverflow.com_questions_17730905_is-there-a-way-to-delete-all-the-data-from-a-topic-or-delete-the-topic-before-ev]

1.  stop zookeeper & Kafka server,
2.  then go to ‘kafka-logs’ folder , there you will see list of kafka topic folders, delete folder with topic name
3.  go to ‘zookeeper-data’ folder , delete data inside that.
4.  start zookeeper & kafka server again.

2， [https://cwiki.apache.org/confluence/display/KAFKA/FAQ\#FAQ-Isitpossibletodeleteatopic][https_cwiki.apache.org_confluence_display_KAFKA_FAQ_FAQ-Isitpossibletodeleteatopic]?  
Deleting a topic is supported since 0.8.2.x. You will need to enable topic deletion (setting delete.topic.enable to true) on all brokers first.

3， [https://cwiki.apache.org/confluence/display/KAFKA/KIP-162+-+Enable+topic+deletion+by+default][https_cwiki.apache.org_confluence_display_KAFKA_KIP-162_-_Enable_topic_deletion_by_default]

4， [https://github.com/darrenfu/bigdata/issues/6][https_github.com_darrenfu_bigdata_issues_6]

\*\*\* The only way to delete a topic permanently is as follows: \*\*\*

stop the brokers
    sudo supervisorctl stop kafka-broker
    remove the directories on disk
    sudo rm -rf <kafka_data_dir>/<topic_name>*
    remove the topic from zookeeper:
    bin/zkCli.sh

rmr /config/topics/<topic\_name>  
rmr /brokers/topics/<topic\_name>  
rmr /admin/delete\_topics/<topic\_name>

verify the topic directory is deleted in zookeeper:

start the brokers back up
    sudo supervisorctl start kafka-broker
    re-create the kafka topic if needed

[70]: /images/20220510/a0f83cedac2f4b739de3ffe3e289ec98.png
[https_stackoverflow.com_questions_17730905_is-there-a-way-to-delete-all-the-data-from-a-topic-or-delete-the-topic-before-ev]: https://stackoverflow.com/questions/17730905/is-there-a-way-to-delete-all-the-data-from-a-topic-or-delete-the-topic-before-ev
[https_cwiki.apache.org_confluence_display_KAFKA_FAQ_FAQ-Isitpossibletodeleteatopic]: https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Isitpossibletodeleteatopic
[https_cwiki.apache.org_confluence_display_KAFKA_KIP-162_-_Enable_topic_deletion_by_default]: https://cwiki.apache.org/confluence/display/KAFKA/KIP-162+-+Enable+topic+deletion+by+default
[https_github.com_darrenfu_bigdata_issues_6]: https://github.com/darrenfu/bigdata/issues/6