安装Zookeeper和Kafka集群

向右看齐 2023-09-26 20:58 41阅读 0赞

## 安装Zookeeper和Kafka集群 ##

本文介绍如何安装Zookeeper和Kafka集群。为了方便，介绍的是在一台服务器上的安装，实际应该安装在多台服务器上，但步骤是一样的。

### 安装Zookeeper集群 ###

#### 下载安装包 ####

从官网上下载安装包：

curl https://dlcdn.apache.org/zookeeper/zookeeper-3.7.1/apache-zookeeper-3.7.1-bin.tar.gz -o apache-zookeeper-3.7.1-bin.tar.gz

解压：

tar xvf apache-zookeeper-3.7.1-bin.tar.gz

#### 配置 ####

创建目录 `zk1`，然后添加如下配置：

`zk1/myid`:

`zk1/zk.config`:

tickTime=2000
    initLimit=10
    syncLimit=5
    dataDir=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/zookeeper/zk1
    clientPort=2181
    
    server.1=127.0.0.1:2888:3888
    server.2=127.0.0.1:2889:3889
    server.3=127.0.0.1:2890:3890

对于`zk2`和`zk3`也重复同样的步骤，并修改相应的配置：

`zk2/myid`:

`zk2/zk.config`:

tickTime=2000
    initLimit=10
    syncLimit=5
    dataDir=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/zookeeper/zk2
    clientPort=2182
    
    server.1=127.0.0.1:2888:3888
    server.2=127.0.0.1:2889:3889
    server.3=127.0.0.1:2890:3890

`zk3/myid`:

`zk3/zk.config`:

tickTime=2000
    initLimit=10
    syncLimit=5
    dataDir=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/zookeeper/zk3
    clientPort=2183
    
    server.1=127.0.0.1:2888:3888
    server.2=127.0.0.1:2889:3889
    server.3=127.0.0.1:2890:3890

#### 启动集群 ####

启动三个服务如下：

$ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh start ./zk1/zk.config 
    ZooKeeper JMX enabled by default
    Using config: ./zk1/zk.config
    Starting zookeeper ... STARTED
    
    $ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh start ./zk2/zk.config 
    ZooKeeper JMX enabled by default
    Using config: ./zk2/zk.config
    Starting zookeeper ... STARTED
    
    $ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh start ./zk3/zk.config 
    ZooKeeper JMX enabled by default
    Using config: ./zk3/zk.config
    Starting zookeeper ... STARTED

#### 查看状态 ####

通过`status`命令查看：

$ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh status ./zk1/zk.config 
    ZooKeeper JMX enabled by default
    Using config: ./zk1/zk.config
    Client port found: 2181. Client address: localhost. Client SSL: false.
    Mode: follower
    
    
    $ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh status ./zk2/zk.config 
    ZooKeeper JMX enabled by default
    Using config: ./zk2/zk.config
    Client port found: 2182. Client address: localhost. Client SSL: false.
    Mode: leader
    
    
    $ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh status ./zk3/zk.config 
    ZooKeeper JMX enabled by default
    Using config: ./zk3/zk.config
    Client port found: 2183. Client address: localhost. Client SSL: false.
    Mode: follower

连接其中一个服务并添加数据：

$ ./apache-zookeeper-3.7.1-bin/bin/zkCli.sh -server localhost:2181
    
    [zk: localhost:2181(CONNECTED) 0] create /pkslow
    Created /pkslow
    [zk: localhost:2181(CONNECTED) 1] create /pkslow/website www.pkslow.com
    Created /pkslow/website

连接另外一个服务，并查看数据，发现与之前创建的是一样的：

$ ./apache-zookeeper-3.7.1-bin/bin/zkCli.sh -server localhost:2182
    
    [zk: localhost:2182(CONNECTED) 1] get /pkslow/website
    www.pkslow.com

目录结构如下：

![a575b5c0ad1286609de07eadb891e922.png][]

### 安装Kafka集群 ###

#### 下载安装包 ####

通过官网下载如下：

curl https://downloads.apache.org/kafka/3.4.0/kafka_2.13-3.4.0.tgz -o kafka_2.13-3.4.0.tgz

解压安装包：

tar -xzf kafka_2.13-3.4.0.tgz

#### 配置 ####

`Broker 1`的配置如下：

broker.id=1
    port=9091
    listeners=PLAINTEXT://:9091
    zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183
    log.dirs=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/kafka/kafka1/kafka-logs

`Broker 2`的配置如下：

broker.id=2
    port=9092
    listeners=PLAINTEXT://:9092
    zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183
    log.dirs=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/kafka/kafka2/kafka-logs

`Broker 3`的配置如下：

broker.id=3
    port=9093
    listeners=PLAINTEXT://:9093
    zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183
    log.dirs=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/kafka/kafka3/kafka-logs

目录结构如下：

![8e79c51d087a7087490fce9f5a3f91aa.png][]

#### 启动集群 ####

启动`kafka`服务如下：

./kafka_2.13-3.4.0/bin/kafka-server-start.sh ./kafka1/server.properties
    ./kafka_2.13-3.4.0/bin/kafka-server-start.sh ./kafka2/server.properties
    ./kafka_2.13-3.4.0/bin/kafka-server-start.sh ./kafka3/server.properties

#### 检查如测试 ####

创建topic：

$ kafka_2.13-3.4.0/bin/kafka-topics.sh --create --topic pkslow-topic --bootstrap-server localhost:9091,localhost:9092,localhost:9093 --partitions 3 --replication-factor 3
    Created topic pkslow-topic.

列出topic：

$ kafka_2.13-3.4.0/bin/kafka-topics.sh --list --bootstrap-server localhost:9091,localhost:9092,localhost:9093
    pkslow-topic

查看topic：

$ kafka_2.13-3.4.0/bin/kafka-topics.sh --describe --topic pkslow-topic --bootstrap-server localhost:9091,localhost:9092,localhost:9093
    Topic: pkslow-topic     TopicId: 7CLy7iZeRvm8rCrn8Dw_mA PartitionCount: 3       ReplicationFactor: 3    Configs: 
            Topic: pkslow-topic     Partition: 0    Leader: 3       Replicas: 3,1,2 Isr: 3,1,2
            Topic: pkslow-topic     Partition: 1    Leader: 1       Replicas: 1,2,3 Isr: 1,2,3
            Topic: pkslow-topic     Partition: 2    Leader: 2       Replicas: 2,3,1 Isr: 2,3,1

生产者发消息到brokers：

$ kafka_2.13-3.4.0/bin/kafka-console-producer.sh --broker-list localhost:9091,localhost:9092,localhost:9093 --topic pkslow-topic
    >My name is Larry Deng.
    >My website is www.pkslow.com.
    >

消费者从brokers收消息：

$ kafka_2.13-3.4.0/bin/kafka-console-consumer.sh --bootstrap-server localhost:9091,localhost:9092,localhost:9093 --topic pkslow-topic --from-beginning
    My name is Larry Deng.
    My website is www.pkslow.com.

###  ###

[a575b5c0ad1286609de07eadb891e922.png]: https://img-blog.csdnimg.cn/img_convert/a575b5c0ad1286609de07eadb891e922.png
[8e79c51d087a7087490fce9f5a3f91aa.png]: https://img-blog.csdnimg.cn/img_convert/8e79c51d087a7087490fce9f5a3f91aa.png