06. Redis 环境搭建-分片集群Cluster

阳光穿透心脏的1/2处 2022-02-02 09:51 312阅读 0赞

Redis 支持分片集群模式, 会将数据散落在不同节点之上, 且每个数据节点都有相应的副本集. 在Redis 3.x 时创建Redis 分片集群还比较麻烦, 需要借助于redis 源码中的ruby 脚本来创建，依赖ruby 环境。 而从Redis 5.x开始, , 创建Redis 分片集群不再支持ruby脚本创建, 而是使用redis-cli 命令直接创建。这种创建方式更为方便.

## 1. 集群配置文件模板 ##

笔者将redis 集群配置存放在/usr/local/etc/redis 目录下

*  从redis源码目录下复制redis.conf 配置文件到/usr/local/etc/redis 目录下
 *  创建集群配置模板文件: /usr/local/etc/redis/template-cluster.conf
 *  新增节点时，只需要替换文件中的6379 即可.

# 引入原配置
    include /usr/local/etc/redis/redis.conf
    
    ######################################## 自定义配置 ########################################
    
    # 修改绑定ip
    bind 127.0.0.1
    
    # 设置关闭保护模式
    protected-mode no
    
    # 设置端口号为6379
    port 6379
    
    # 设置后台启动为yes
    daemonize yes
    
    # 设置pid文件
    pidfile /logs/redis/redis.pid.6379
    
    # 设置日志文件
    logfile /logs/redis/redis.log.6379
    
    # 设置RDB dum文件名
    dbfilename dump.rdb.6379
    
    # 设置dump文件存放目录
    dir /data/redis
    
    # 设置主节点密码
    masterauth 123456
    
    # 设置从节点只读
    replica-read-only yes
    
    # 设置当前节点访问密码为
    requirepass 123456
    
    # 设置客户端最多连接数
    maxclients 1000
    
    # 设置最大内存
    maxmemory 2GB
    
    # 设置达到最大内存后策略，报错
    maxmemory-policy noeviction
    
    # 启动AOF持久化模式
    appendonly yes
    
    # 设置AOF文件名
    appendfilename "appendonly.aof.6379"
    
    # 设置AOF写入磁盘策略, 每秒写入一次
    appendfsync everysec
    
    # 开启集群
    cluster-enabled yes
    
    # 集群节点连接超时时间
    cluster-node-timeout 15000  
    
    # 集群节点配置信息文件, 位于redis 数据文件存放目录，即$dir 定义的目录中
    cluster-config-file nodes.conf.6379

## 2. 创建集群配置 ##

对于Redis 分片集群, 笔者认为, 集群节点至少应该有9个。 三个主节点, 6个从节点. 这样集群才能集高可用和分片于一身. 笔者创建一个9个节点的集群, 端口号为：7481~7489

#### 2.1 复制9个节点配置 ####

配置文件均存放在/usr/local/etc/redis目录.

cp template-cluster.conf redis.conf.7481
    cp template-cluster.conf redis.conf.7482
    cp template-cluster.conf redis.conf.7483
    cp template-cluster.conf redis.conf.7484
    cp template-cluster.conf redis.conf.7485
    cp template-cluster.conf redis.conf.7486
    cp template-cluster.conf redis.conf.7487
    cp template-cluster.conf redis.conf.7488
    cp template-cluster.conf redis.conf.7489

### 2.2 替换端口号 ###

依次替换每个节点配置的端口号，日志文件，dump文件等配置.

sed -i "s/6379/7481/g" redis.conf.7481
    sed -i "s/6379/7482/g" redis.conf.7482
    sed -i "s/6379/7483/g" redis.conf.7483
    sed -i "s/6379/7484/g" redis.conf.7484
    sed -i "s/6379/7485/g" redis.conf.7485
    sed -i "s/6379/7486/g" redis.conf.7486
    sed -i "s/6379/7487/g" redis.conf.7487
    sed -i "s/6379/7488/g" redis.conf.7488
    sed -i "s/6379/7489/g" redis.conf.7489

## 3. 启动redis 集群 ##

### 3.1 启动redis ###

依次启动9台服务器!

redis-server redis.conf.7481
    redis-server redis.conf.7482
    redis-server redis.conf.7483
    redis-server redis.conf.7484
    redis-server redis.conf.7485
    redis-server redis.conf.7486
    redis-server redis.conf.7487
    redis-server redis.conf.7488
    redis-server redis.conf.7489

### 3.2 创建集群关系 ###

使用redis-cli --cluster 命令建立集群关系， 需要注意的是，每个redis 都必须是空的, 即是新建的节点, 否则将无法创建。

$ redis-cli -a 123456 --cluster create \
    > 127.0.0.1:7481 \
    > 127.0.0.1:7482 \
    > 127.0.0.1:7483 \
    > 127.0.0.1:7484 \
    > 127.0.0.1:7485 \
    > 127.0.0.1:7486 \
    > 127.0.0.1:7487 \
    > 127.0.0.1:7488 \
    > 127.0.0.1:7489 \
    > --cluster-replicas 2
    
    
    Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
    >>> Performing hash slots allocation on 9 nodes...
    Master[0] -> Slots 0 - 5460
    Master[1] -> Slots 5461 - 10922
    Master[2] -> Slots 10923 - 16383
    Adding replica 127.0.0.1:7485 to 127.0.0.1:7481
    Adding replica 127.0.0.1:7486 to 127.0.0.1:7481
    Adding replica 127.0.0.1:7487 to 127.0.0.1:7482
    Adding replica 127.0.0.1:7488 to 127.0.0.1:7482
    Adding replica 127.0.0.1:7489 to 127.0.0.1:7483
    Adding replica 127.0.0.1:7484 to 127.0.0.1:7483
    >>> Trying to optimize slaves allocation for anti-affinity
    [WARNING] Some slaves are in the same host as their master
    M: 9baa3a6f75fe188460d337cfa8d6df862412e6e4 127.0.0.1:7481
       slots:[0-5460] (5461 slots) master
    M: 829f7c60a21a5033ce88f0c7e54b17083ca6a691 127.0.0.1:7482
       slots:[5461-10922] (5462 slots) master
    M: 97324a77a0ebfa3ac237abfd2593347b93aad719 127.0.0.1:7483
       slots:[10923-16383] (5461 slots) master
    S: 1362c9a5988eba9a227595670b3e98f70767935a 127.0.0.1:7484
       replicates 829f7c60a21a5033ce88f0c7e54b17083ca6a691
    S: 7086538d6d4c11a5a52580082c4fd963814365cd 127.0.0.1:7485
       replicates 9baa3a6f75fe188460d337cfa8d6df862412e6e4
    S: 3a5f4c13439bc20586238ef615e27a91dc47b251 127.0.0.1:7486
       replicates 9baa3a6f75fe188460d337cfa8d6df862412e6e4
    S: c36937c4281e8f891e88b02102fe2f59c15d4769 127.0.0.1:7487
       replicates 97324a77a0ebfa3ac237abfd2593347b93aad719
    S: 3865b037f0026306de024629a4b7298e9c80d4c9 127.0.0.1:7488
       replicates 97324a77a0ebfa3ac237abfd2593347b93aad719
    S: 869576474847e228be3d9d5377b01dd3040b1cae 127.0.0.1:7489
       replicates 829f7c60a21a5033ce88f0c7e54b17083ca6a691
    Can I set the above configuration? (type 'yes' to accept): yes
    >>> Nodes configuration updated
    >>> Assign a different config epoch to each node
    >>> Sending CLUSTER MEET messages to join the cluster
    Waiting for the cluster to join
    ....
    >>> Performing Cluster Check (using node 127.0.0.1:7481)
    M: 9baa3a6f75fe188460d337cfa8d6df862412e6e4 127.0.0.1:7481
       slots:[0-5460] (5461 slots) master
       2 additional replica(s)
    S: 7086538d6d4c11a5a52580082c4fd963814365cd 127.0.0.1:7485
       slots: (0 slots) slave
       replicates 9baa3a6f75fe188460d337cfa8d6df862412e6e4
    S: c36937c4281e8f891e88b02102fe2f59c15d4769 127.0.0.1:7487
       slots: (0 slots) slave
       replicates 97324a77a0ebfa3ac237abfd2593347b93aad719
    S: 3a5f4c13439bc20586238ef615e27a91dc47b251 127.0.0.1:7486
       slots: (0 slots) slave
       replicates 9baa3a6f75fe188460d337cfa8d6df862412e6e4
    S: 3865b037f0026306de024629a4b7298e9c80d4c9 127.0.0.1:7488
       slots: (0 slots) slave
       replicates 97324a77a0ebfa3ac237abfd2593347b93aad719
    M: 97324a77a0ebfa3ac237abfd2593347b93aad719 127.0.0.1:7483
       slots:[10923-16383] (5461 slots) master
       2 additional replica(s)
    S: 869576474847e228be3d9d5377b01dd3040b1cae 127.0.0.1:7489
       slots: (0 slots) slave
       replicates 829f7c60a21a5033ce88f0c7e54b17083ca6a691
    S: 1362c9a5988eba9a227595670b3e98f70767935a 127.0.0.1:7484
       slots: (0 slots) slave
       replicates 829f7c60a21a5033ce88f0c7e54b17083ca6a691
    M: 829f7c60a21a5033ce88f0c7e54b17083ca6a691 127.0.0.1:7482
       slots:[5461-10922] (5462 slots) master
       2 additional replica(s)
    [OK] All nodes agree about slots configuration.
    >>> Check for open slots...
    >>> Check slots coverage...
    [OK] All 16384 slots covered.

### 3.3 查看集群状态 ###

通过cluster nodes 命令, 可以看出redis各节点主从关系.

$ redis-cli -a 123456 -p 7481 cluster nodes
    Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
    1c9ee4a2f7e0191983f67c8849e61f8cd5c91005 127.0.0.1:7482@17482 master - 0 1557385770000 2 connected 5461-10922
    06993924bd468b6eddb20be332b762a956b6631b 127.0.0.1:7489@17489 slave 1f5f8ba346034004cdd52ef0f267f024b1be6ee3 0 1557385771602 9 connected
    7229f95a77bb60065a78e3dc422c056e265b05b3 127.0.0.1:7485@17485 slave 57fb7a0de46ef6340ece8f6a67829c49aeb91c96 0 1557385769000 5 connected
    e25d2f1a019a40282e3c9146a00e44f4aa077afa 127.0.0.1:7486@17486 slave 1c9ee4a2f7e0191983f67c8849e61f8cd5c91005 0 1557385770599 6 connected
    1f5f8ba346034004cdd52ef0f267f024b1be6ee3 127.0.0.1:7483@17483 master - 0 1557385768000 3 connected 10923-16383
    7464b7aafc58e9235fc2a5c40f999018bb8b5509 127.0.0.1:7484@17484 slave 57fb7a0de46ef6340ece8f6a67829c49aeb91c96 0 1557385768000 4 connected
    fe1a76b8052de772ed3aa9472726507ee48a5f1d 127.0.0.1:7488@17488 slave 1c9ee4a2f7e0191983f67c8849e61f8cd5c91005 0 1557385769000 8 connected
    57fb7a0de46ef6340ece8f6a67829c49aeb91c96 127.0.0.1:7481@17481 myself,master - 0 1557385769000 1 connected 0-5460
    e72fb434274ce30e0da21130e9279c5a700419b6 127.0.0.1:7487@17487 slave 1f5f8ba346034004cdd52ef0f267f024b1be6ee3 0 1557385769596 7 connected