SparkStream+kafka报错Fetching topic metadata with correlation id 0 for topics from broker

本是古典何须时尚 2023-05-31 03:17 12阅读 0赞

19/10/22 17:35:44 WARN ClientUtils$: Fetching topic metadata with correlation id 0 
    for topics [Set(zhaogw)] from broker [id:0,host:dblab-VirtualBox,port:9092] failed
    java.nio.channels.ClosedChannelException
    	at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
    	at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)

先说解决方法，因为我的主机名称为`dblab-VirtualBox`，而在我本机配置虚拟机的映射为bigdata，因为而kafka默认拿着主机名寻找映射信息，然而hosts文件中没有，故报错。  
**如果还不懂，看图：**

#### 修改前 ####

这是我配置的linux主机名  
![在这里插入图片描述][20191022174553438.png]  
这是我在windows的hosts文件所做的映射  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg1NDg4_size_16_color_FFFFFF_t_70]  
代码

val kafkaDsream: ReceiverInputDStream[(String, String)] = KafkaUtils.createStream(
          streamContext,
          "bigdata:2181",
          "zhaogw",
          Map("zhaogw" -> 3)
        )

启动项目后就不断的刷那个异常

#### 修改后 ####

wondows的hosts文件  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg1NDg4_size_16_color_FFFFFF_t_70 1]  
代码也改一下  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg1NDg4_size_16_color_FFFFFF_t_70 2]  
项目就能正常启动消费了

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg1NDg4_size_16_color_FFFFFF_t_70 3]

参考[https://www.cnblogs.com/dongxiucai/p/10288275.html][https_www.cnblogs.com_dongxiucai_p_10288275.html]

[20191022174553438.png]: /images/20230531/384d4d6d8fcc4f40aaa6d0f5022072d9.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg1NDg4_size_16_color_FFFFFF_t_70]: /images/20230531/1acad246b987498a828af7179571e215.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg1NDg4_size_16_color_FFFFFF_t_70 1]: /images/20230531/e6e77f978c3641dcb0a87b425ef2e8ba.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg1NDg4_size_16_color_FFFFFF_t_70 2]: /images/20230531/d5d350bca6f6449ab1e21fe796ed74b5.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg1NDg4_size_16_color_FFFFFF_t_70 3]: /images/20230531/0c4fc0a85a6e41489868d93a1db7c8e9.png
[https_www.cnblogs.com_dongxiucai_p_10288275.html]: https://www.cnblogs.com/dongxiucai/p/10288275.html