kafka单机版环境搭建

迈不过友情╰ 2023-10-06 17:33 66阅读 0赞

#### 前言 ####

最近把kafak的数据导入到clickhouse，所以安装一个单机版的kakfa感受下。

#### 1.zookeeper安装 ####

要装kafka，它依赖zookeeper，所以要先安装zookpeer。  
zookeeper 下载  
[https://zookeeper.apache.org/releases.html][https_zookeeper.apache.org_releases.html]

wegt https://mirrors.bfsu.edu.cn/apache/zookeeper/zookeeper-3.6.3/apache-zookeeper-3.6.3.tar.gz

在能文件下 去执行 就下载到那个文件下

然后解压

tar -zxf apache-zookeeper-3.6.3-bin.tar.gz

![在这里插入图片描述][20210430202501339.png]

cd apache-zookeeper-3.6.3-bin  
在这个文件下 创建 数据存储和 日志存储的路径  
mkdir data  
mkdir logs

然后再 cd apache-zookeeper-3.6.3-bin/conf  
　cp zoo\_sample.cfg zoo.cfg  
　vi zoo.cfg  
　  
添加这个配置  
dataDir=/usr/local/work/zookeeper/apache-zookeeper-3.6.3-bin/data  
dataLogDir=/usr/local/work/zookeeper/apache-zookeeper-3.6.3-bin/logs

生效  
source zoo.cfg

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80Mzk3NTc3MQ_size_16_color_FFFFFF_t_70]  
jvm 环境变量

vim /etc/profile

ZOOKEEPER_HOME=/usr/local/work/zookeeper/apache-zookeeper-3.6.3-bin
    PATH=$PATH:$ZOOKEEPER_HOME/bin
    export PATH ZOOKEEPER_HOME PATH

sourc /etc/profile

启动zookeeper。

/usr/local/worl/zookeeper/apache-zookeeper-3.6.3-bin/bin/zkServer.sh start

zk启动

1.  启动ZK服务: bin/zkServer.sh start
2.  查看ZK服务状态: bin/zkServer.sh status
3.  停止ZK服务: bin/zkServer.sh stop
4.  重启ZK服务: bin/zkServer.sh restart
5.  连接服务器: zkCli.sh -server 127.0.0.1:2181

#### 2. 安装kafka ####

kafka 下载 [http://kafka.apache.org/downloads][http_kafka.apache.org_downloads]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80Mzk3NTc3MQ_size_16_color_FFFFFF_t_70 1]  
也可以这样

wget https://archive.apache.org/dist/kafka/1.1.0/kafka_2.11-1.1.0.tgz

配置 jvm  
vi /etc/profile

export KAFKA_HOME=/usr/local/work/kafka/kafka_2.11-0.10.1.1
    PATH=$PATH:$HOME/bin:$KAFKA_HOME/bin

source /etc/profile

要去kafka 安装的 config 文件找到 server.properties

log.dirs=/export/server/kafka/logs

配置这个是为了生产和消费之间的通信

advertised.host.name=192.168.208.136  
advertised.port=9092

# 解压缩文件
    tar zxvf kafka_2.11-1.1.0.tgz
    cd kafka_2.11-1.1.0/
    # 启动zk
    bin/zookeeper-server-start.sh -daemon config/zookeeper.properties
    # 检查zk是否启动成功
    netstat -tunpl|grep 2181
    tcp6       0      0 :::2181                 :::*                    LISTEN      2877/java    
    # 启动kafka
    bin/kafka-server-start.sh -daemon ../config/server.properties 
    # 检查kafka是否启动成功
    netstat -tunpl|grep 9092
    tcp6       0      0 :::9092                 :::*                    LISTEN      3164/java

打开一个窗口，输入如下的指令，启动生产者：

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test

1.  运行 consumer

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning

注意 出现错误：  
zookeeper is not a recognized option zookeeper参数不支持

**解决办法：**

1.继续使用新版本

./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning

2.Kafka更换成老版本，并使用 --zookeeper

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning

[https_zookeeper.apache.org_releases.html]: https://zookeeper.apache.org/releases.html
[20210430202501339.png]: https://img-blog.csdnimg.cn/20210430202501339.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80Mzk3NTc3MQ_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20210430201916879.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80Mzk3NTc3MQ==,size_16,color_FFFFFF,t_70
[http_kafka.apache.org_downloads]: http://kafka.apache.org/downloads
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80Mzk3NTc3MQ_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20210430203201329.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80Mzk3NTc3MQ==,size_16,color_FFFFFF,t_70