Flume自定义开发常见异常及解决办法

蔚落 2022-05-23 10:20 907阅读 0赞

### 前言 ###

前些日子，对Flume进行了自定义sink和source的开发。遇到一些异常，以后关于Flume的异常都统一记录与此处。(除了个别较为麻烦的异常处理)

--------------------

### 正文 ###

*  Flume source has been removed due to an error in configuration  
    中文含义: Flume的source部分因为配置异常被移除。  
    表象: Flume中source没有启动。  
    解决办法：查看上下文，查看诸如`java.lang.IllegalStateException: The parameter command must be specified`类似的具体异常，具体定位问题。(一般是配置文件或者自定义source未写正确导致，可以try-catch捕获处理某些异常。)

05 六月 2018 16:52:57,528 ERROR [conf-file-poller-0] (org.apache.flume.node.AbstractConfigurationProvider.loadSinks:426)  - Sink epp has been removed due to an error during configuration
    org.apache.flume.conf.ConfigurationException: Bootstrap Servers must be specified
    	at org.apache.flume.sink.kafka.KafkaSink.translateOldProps(KafkaSink.java:353)
    	at org.apache.flume.sink.kafka.KafkaSink.configure(KafkaSink.java:295)
    	at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
    	at org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:411)
    	at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:102)
    	at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:141)
    	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    	at java.lang.Thread.run(Thread.java:745)

*  Flume agent unable to deliver event.  
    分析: 和上个异常类似。 一般跟有`ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.Sink`类似的报错。

### Reference ###

\[1\]. [(stackoverflow)flume-source-has-been-removed-due-to-an-error-in-configuration][stackoverflow_flume-source-has-been-removed-due-to-an-error-in-configuration]  
\[2\] [(stackover)flume-agent-unable-to-deliver-event][stackover_flume-agent-unable-to-deliver-event]

[stackoverflow_flume-source-has-been-removed-due-to-an-error-in-configuration]: https://stackoverflow.com/questions/43662161/flume-source-has-been-removed-due-to-an-error-in-configuration
[stackover_flume-agent-unable-to-deliver-event]: https://stackoverflow.com/questions/20745439/flume-agent-unable-to-deliver-event