Flume 配置 Kafka Channel 啟動報錯解決方案

在虛擬機環境中搭建 Flume 與 Kafka 整合時，Flume 部署於 172.235.10.10，Kafka 部署於 172.235.10.11。配置文件如下：

Flume 配置詳情

a1.sources = r1
a1.sinks = k1
a1.channels = c1

a1.sources.r1.type = http
a1.sources.r1.channels = c1
a1.sources.r1.bind = 0.0.0.0
a1.sources.r1.port = 8000

a1.sinks.k1.type = file_roll
a1.sinks.k1.channel = c1
a1.sinks.k1.directory = /usr/log/flume

a1.channels.c1.type = org.apache.flume.channel.kafka.KafkaChannel
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
a1.channels.c1.brokerList = 172.235.10.11:9002
a1.channels.c1.topic = test2
a1.channels.c1.zookeeperConnect = 172.235.10.11:2181

Kafka 配置

ZooKeeper 使用 Kafka 自帶版本，未修改配置。Kafka server.properties 修改如下：

host.name = 172.235.10.11
zookeeper.connect = 172.235.10.11:2181

Kafka 啟動正常，生產者與消費者測試通訊無誤。

啟動報錯信息

但啟動 Flume 時出現錯誤：

Error while getting events from Kafka. This is usually caused by trying to read a non-flume event. Ensure the setting for parseAsFlumeEvent is correct

解決思路

此錯誤通常因 Kafka Channel 讀取非 Flume 事件格式導致。建議檢查 parseAsFlumeEvent 參數設置，或嘗試配置多個 Kafka Source 從同一 Topic 讀取。可參考相關技術博客深入排查。