Redis【性能 02】Redis-5.0.14伪集群和Docker集群搭建及延迟和性能测试（均无法提升性能）

落日映苍穹つ 2023-09-27 18:42 107阅读 0赞

### 伪集群及Docker集群搭建测试流程 ###

*  1.伪集群搭建
 *   *  1.1 环境
     *  1.2 搭建
     *   *  1.2.1 集群配置
         *  1.2.2 生成其他5个节点配置
         *  1.2.3 启动并验证节点状态
         *  1.2.4 创建集群
         *  1.2.5 集群信息
     *  1.3 测试
 *  2.Docker集群
 *   *  2.1 环境
     *  2.2 搭建
     *   *  2.2.1 创建专用网络
         *  2.2.2 生成配置文件
         *  2.2.3 容器启动及验证
         *  2.2.4 创建集群
         *  2.2.5 删除容器及配置文件
     *  2.3.测试
 *  3.总结

# 1.伪集群搭建 #

## 1.1 环境 ##

使用的是腾讯的云服务器`1核心2G内存50G存储`，系统信息如下：

[root@tcloud ~]# rpm -qi centos-release
    Name        : centos-release
    Version     : 7
    Release     : 9.2009.1.el7.centos
    Architecture: x86_64
    Install Date: Tue 19 Jan 2021 06:23:16 PM CST
    Group       : System Environment/Base
    Size        : 44787
    License     : GPLv2
    Signature   : RSA/SHA256, Thu 03 Dec 2020 12:35:28 AM CST, Key ID 24c6a8a7f4a80eb5
    Source RPM  : centos-release-7-9.2009.1.el7.centos.src.rpm
    Build Date  : Mon 23 Nov 2020 11:08:41 PM CST
    Build Host  : x86-01.bsys.centos.org
    Relocations : (not relocatable)
    Packager    : CentOS BuildSystem <http://bugs.centos.org>
    Vendor      : CentOS
    Summary     : CentOS Linux release file
    Description :
    CentOS Linux release files

## 1.2 搭建 ##

*  每个Redis节点必须要有一个备机，例如搭建3个节点的集群就要有6个Redis实例。
 *  数据按照slots分布式存储在不同的Redis节点上，节点中的数据可共享，可以动态调整数据的分布。
 *  可扩展性强，可以动态增删节点 ，最多可扩展1000+节点。

### 1.2.1 集群配置 ###

使用相对性能较好的`5.0.14`版本进行搭建：

# 1.复制单机版并修改配置
    cp -r ./redis-5.0.14 ./redis-5.0.14-0
    
    # 修改【redis-5.0.14-0】的配置信息如下
    ################################## NETWORK #####################################
    protected-mode no
    port 6370
    ################################# GENERAL #####################################
    daemonize yes
    ################################ REDIS CLUSTER  ###############################
    cluster-enabled yes
    cluster-config-file nodes-6370.conf
    cluster-node-timeout 15000

### 1.2.2 生成其他5个节点配置 ###

# 2.复制5个节点并修改配置
    cp -r ./redis-5.0.14-0 ./redis-5.0.14-1
    cp -r ./redis-5.0.14-0 ./redis-5.0.14-2
    cp -r ./redis-5.0.14-0 ./redis-5.0.14-3
    cp -r ./redis-5.0.14-0 ./redis-5.0.14-4
    cp -r ./redis-5.0.14-0 ./redis-5.0.14-5
    
    # 修改每个节点的配置
    sed 's/6370/6371/g' ./redis-5.0.14-1/redis.conf > ./redis-5.0.14-1/redis-6371.conf
    sed 's/6370/6372/g' ./redis-5.0.14-2/redis.conf > ./redis-5.0.14-2/redis-6372.conf
    sed 's/6370/6373/g' ./redis-5.0.14-3/redis.conf > ./redis-5.0.14-3/redis-6373.conf
    sed 's/6370/6374/g' ./redis-5.0.14-4/redis.conf > ./redis-5.0.14-4/redis-6374.conf
    sed 's/6370/6375/g' ./redis-5.0.14-5/redis.conf > ./redis-5.0.14-5/redis-6375.conf

### 1.2.3 启动并验证节点状态 ###

# 3.启动6个节点并验证启动状态
    ./redis-5.0.14-0/bin/redis-server ./redis-5.0.14-0/redis.conf
    ./redis-5.0.14-1/bin/redis-server ./redis-5.0.14-1/redis-6371.conf
    ./redis-5.0.14-2/bin/redis-server ./redis-5.0.14-2/redis-6372.conf
    ./redis-5.0.14-3/bin/redis-server ./redis-5.0.14-3/redis-6373.conf
    ./redis-5.0.14-4/bin/redis-server ./redis-5.0.14-4/redis-6374.conf
    ./redis-5.0.14-5/bin/redis-server ./redis-5.0.14-5/redis-6375.conf
    
    [root@tcloud local]# ps -ef | grep redis
    root      1866     1  0 17:12 ?        00:00:00 ./redis-5.0.14-0/bin/redis-server 127.0.0.1:6370 [cluster]
    root      1871     1  0 17:12 ?        00:00:00 ./redis-5.0.14-1/bin/redis-server 127.0.0.1:6371 [cluster]
    root      1876     1  0 17:12 ?        00:00:00 ./redis-5.0.14-2/bin/redis-server 127.0.0.1:6372 [cluster]
    root      1881     1  0 17:12 ?        00:00:00 ./redis-5.0.14-3/bin/redis-server 127.0.0.1:6373 [cluster]
    root      1886     1  0 17:12 ?        00:00:00 ./redis-5.0.14-4/bin/redis-server 127.0.0.1:6374 [cluster]
    root      1896     1  0 17:12 ?        00:00:00 ./redis-5.0.14-5/bin/redis-server 127.0.0.1:6375 [cluster]
    root      1957  1522  0 17:12 pts/0    00:00:00 grep --color=auto redis

### 1.2.4 创建集群 ###

# 4.创建集群
    [root@tcloud redis-5.0.14-0]# ./bin/redis-cli --cluster create 127.0.0.1:6370 127.0.0.1:6371 127.0.0.1:6372 127.0.0.1:6373 127.0.0.1:6374 127.0.0.1:6375 --cluster-replicas 1
    >>> Performing hash slots allocation on 6 nodes...
    Master[0] -> Slots 0 - 5460
    Master[1] -> Slots 5461 - 10922
    Master[2] -> Slots 10923 - 16383
    Adding replica 127.0.0.1:6374 to 127.0.0.1:6370
    Adding replica 127.0.0.1:6375 to 127.0.0.1:6371
    Adding replica 127.0.0.1:6373 to 127.0.0.1:6372
    >>> Trying to optimize slaves allocation for anti-affinity
    [WARNING] Some slaves are in the same host as their master
    M: 60772cf691608a8ae8e6f519b1a95b52c84b2457 127.0.0.1:6370
       slots:[0-5460] (5461 slots) master
    M: ffe9663d3fd1fc55c74c7bea6c70069fb742a69f 127.0.0.1:6371
       slots:[5461-10922] (5462 slots) master
    M: 6bb4e92cec0464c9969397eccaf1de1ce54b3e56 127.0.0.1:6372
       slots:[10923-16383] (5461 slots) master
    S: e793d2b868dbcd0b701bae10ea4d689b13a1c650 127.0.0.1:6373
       replicates ffe9663d3fd1fc55c74c7bea6c70069fb742a69f
    S: baae7ed422dc563bbbd22e1ac2d859ba4835dd63 127.0.0.1:6374
       replicates 6bb4e92cec0464c9969397eccaf1de1ce54b3e56
    S: 2fddd068ad1ae78d816d2932f012fae025e4e1f8 127.0.0.1:6375
       replicates 60772cf691608a8ae8e6f519b1a95b52c84b2457
    Can I set the above configuration? (type 'yes' to accept): yes
    >>> Nodes configuration updated
    >>> Assign a different config epoch to each node
    >>> Sending CLUSTER MEET messages to join the cluster
    Waiting for the cluster to join
    .
    >>> Performing Cluster Check (using node 127.0.0.1:6370)
    M: 60772cf691608a8ae8e6f519b1a95b52c84b2457 127.0.0.1:6370
       slots:[0-5460] (5461 slots) master
       1 additional replica(s)
    M: 6bb4e92cec0464c9969397eccaf1de1ce54b3e56 127.0.0.1:6372
       slots:[10923-16383] (5461 slots) master
       1 additional replica(s)
    M: ffe9663d3fd1fc55c74c7bea6c70069fb742a69f 127.0.0.1:6371
       slots:[5461-10922] (5462 slots) master
       1 additional replica(s)
    S: baae7ed422dc563bbbd22e1ac2d859ba4835dd63 127.0.0.1:6374
       slots: (0 slots) slave
       replicates 6bb4e92cec0464c9969397eccaf1de1ce54b3e56
    S: 2fddd068ad1ae78d816d2932f012fae025e4e1f8 127.0.0.1:6375
       slots: (0 slots) slave
       replicates 60772cf691608a8ae8e6f519b1a95b52c84b2457
    S: e793d2b868dbcd0b701bae10ea4d689b13a1c650 127.0.0.1:6373
       slots: (0 slots) slave
       replicates ffe9663d3fd1fc55c74c7bea6c70069fb742a69f
    [OK] All nodes agree about slots configuration.
    >>> Check for open slots...
    >>> Check slots coverage...
    [OK] All 16384 slots covered.

### 1.2.5 集群信息 ###

# 5.集群并查看节点数据
    [root@tcloud redis-5.0.14-0]# ./bin/redis-cli -c -p 6370
    127.0.0.1:6370> cluster nodes
    6bb4e92cec0464c9969397eccaf1de1ce54b3e56 127.0.0.1:6372@16372 master - 0 1683192656954 3 connected 10923-16383
    ffe9663d3fd1fc55c74c7bea6c70069fb742a69f 127.0.0.1:6371@16371 master - 0 1683192657956 2 connected 5461-10922
    60772cf691608a8ae8e6f519b1a95b52c84b2457 127.0.0.1:6370@16370 myself,master - 0 1683192655000 1 connected 0-5460
    baae7ed422dc563bbbd22e1ac2d859ba4835dd63 127.0.0.1:6374@16374 slave 6bb4e92cec0464c9969397eccaf1de1ce54b3e56 0 1683192655951 5 connected
    2fddd068ad1ae78d816d2932f012fae025e4e1f8 127.0.0.1:6375@16375 slave 60772cf691608a8ae8e6f519b1a95b52c84b2457 0 1683192655000 6 connected
    e793d2b868dbcd0b701bae10ea4d689b13a1c650 127.0.0.1:6373@16373 slave ffe9663d3fd1fc55c74c7bea6c70069fb742a69f 0 1683192653000 4 connected

## 1.3 测试 ##

# 延迟测试
    ./bin/redis-cli -c -p 6370 --intrinsic-latency 10
    # 性能测试
    ./bin/redis-benchmark -h 127.0.0.1 -p 6370 -q

*  延迟测试结果

# 1
    150971628 total runs (avg latency: 0.0662 microseconds / 66.24 nanoseconds per run).
    Worst run took 117245x longer than the average latency.
    # 2
    149243181 total runs (avg latency: 0.0670 microseconds / 67.00 nanoseconds per run).
    Worst run took 155735x longer than the average latency.
    # 3
    150936098 total runs (avg latency: 0.0663 microseconds / 66.25 nanoseconds per run).
    Worst run took 87950x longer than the average latency.

*  性能测试结果

# 1
    PING_INLINE: 62814.07
    PING_BULK: 64143.68
    SET: 61087.36
    GET: 62344.14
    INCR: 59665.87
    LPUSH: 54945.05
    RPUSH: 56785.91
    LPOP: 55991.04
    RPOP: 57208.24
    SADD: 62932.66
    HSET: 59347.18
    SPOP: 63091.48
    LPUSH (needed to benchmark LRANGE): 56211.35
    LRANGE_100 (first 100 elements): 34614.05
    LRANGE_300 (first 300 elements): 16131.63
    LRANGE_500 (first 450 elements): 11568.72
    LRANGE_600 (first 600 elements): 9214.89
    MSET (10 keys): 54644.81
    # 2
    PING_INLINE: 61881.19
    PING_BULK: 64267.35
    SET: 61881.19
    GET: 61804.70
    INCR: 61387.36
    LPUSH: 55555.56
    RPUSH: 55370.98
    LPOP: 56369.79
    RPOP: 57736.72
    SADD: 61614.29
    HSET: 61387.36
    SPOP: 63613.23
    LPUSH (needed to benchmark LRANGE): 53792.36
    LRANGE_100 (first 100 elements): 33681.38
    LRANGE_300 (first 300 elements): 15637.22
    LRANGE_500 (first 450 elements): 11828.72
    LRANGE_600 (first 600 elements): 9221.69
    MSET (10 keys): 52826.20
    # 3
    PING_INLINE: 63091.48
    PING_BULK: 63572.79
    SET: 61012.81
    GET: 60569.35
    INCR: 60459.49
    LPUSH: 55463.12
    RPUSH: 56850.48
    LPOP: 55617.35
    RPOP: 57570.52
    SADD: 60790.27
    HSET: 60168.47
    SPOP: 62695.92
    LPUSH (needed to benchmark LRANGE): 55679.29
    LRANGE_100 (first 100 elements): 34435.26
    LRANGE_300 (first 300 elements): 16064.26
    LRANGE_500 (first 450 elements): 11496.90
    LRANGE_600 (first 600 elements): 9180.21
    MSET (10 keys): 53648.07

![在这里插入图片描述][27f233a5ce9d4b2392abf4b180c723b1.png_pic_center]

# 2.Docker集群 #

## 2.1 环境 ##

[root@tcloud ~]# docker -v
    Docker version 20.10.13, build a224086

Redis的官方Docker镜像地址：https://hub.docker.com/\_/redis

## 2.2 搭建 ##

### 2.2.1 创建专用网络 ###

# 创建桥接网卡
    docker network create redis --subnet 172.81.0.0/16
    
    # 查看所有 网卡
    docker network ls
    # 查看网卡详情
    docker network inspect NETWORKID
    
    [
        {
        
            "Name": "redis",
            "Id": "79a62c1104195f5f077665fcdd8d4e1c92697af7db227c1fd1a92219f22c2501",
            "Created": "2023-05-05T10:38:24.604493511+08:00",
            "Scope": "local",
            "Driver": "bridge",
            "EnableIPv6": false,
            "IPAM": {
        
                "Driver": "default",
                "Options": {
        },
                "Config": [
                    {
        
                        "Subnet": "172.81.0.0/16",
                        "Gateway": "172.81.0.1"
                    }
                ]
            },
            "Internal": false,
            "Attachable": false,
            "Ingress": false,
            "ConfigFrom": {
        
                "Network": ""
            },
            "ConfigOnly": false,
            "Containers": {
        },
            "Options": {
        },
            "Labels": {
        }
        }
    ]

### 2.2.2 生成配置文件 ###

# 通过脚本创建6个Redis配置
    # 这里一定要注意不要加 daemonize yes 否则无法启动
    for port in $(seq 1 6); \
    do \
    mkdir -p /usr/local/redis-docker/node-${port}/conf
    touch /usr/local/redis-docker/node-${port}/conf/redis.conf
    cat << EOF >/usr/local/redis-docker/node-${port}/conf/redis.conf
    bind 0.0.0.0
    protected-mode no
    port 637${port}
    cluster-enabled yes
    cluster-config-file nodes.conf
    cluster-node-timeout 5000
    cluster-announce-ip 172.81.0.1${port}
    cluster-announce-port 637${port}
    cluster-announce-bus-port 1637${port}
    appendonly no
    EOF
    done

### 2.2.3 容器启动及验证 ###

# 启动容器
    for port in $(seq 1 6); \
    do \
    docker run -p 637${port}:637${port} -p 1637${port}:1637${port} --name redis-${port} \
    -v /usr/local/redis-docker/node-${port}/data:/data \
    -v /usr/local/redis-docker/node-${port}/conf/redis.conf:/etc/redis/redis.conf \
    -d --net redis --ip 172.81.0.1${port} redis:5.0.14 redis-server /etc/redis/redis.conf
    done
    
    # 查看启动状态
    [root@tcloud local]# docker ps | grep redis
    a4e70ff2d840   redis:5.0.14     "docker-entrypoint.s…"   48 seconds ago   Up 46 seconds       0.0.0.0:6376->6376/tcp, :::6376->6376/tcp, 0.0.0.0:16376->16376/tcp, :::16376->16376/tcp, 6379/tcp   redis-6
    854f8e023ce1   redis:5.0.14     "docker-entrypoint.s…"   49 seconds ago   Up 47 seconds       0.0.0.0:6375->6375/tcp, :::6375->6375/tcp, 0.0.0.0:16375->16375/tcp, :::16375->16375/tcp, 6379/tcp   redis-5
    5165d1e61a7f   redis:5.0.14     "docker-entrypoint.s…"   50 seconds ago   Up 48 seconds       0.0.0.0:6374->6374/tcp, :::6374->6374/tcp, 0.0.0.0:16374->16374/tcp, :::16374->16374/tcp, 6379/tcp   redis-4
    f08dbda27366   redis:5.0.14     "docker-entrypoint.s…"   51 seconds ago   Up 50 seconds       0.0.0.0:6373->6373/tcp, :::6373->6373/tcp, 0.0.0.0:16373->16373/tcp, :::16373->16373/tcp, 6379/tcp   redis-3
    e5e0feb9546a   redis:5.0.14     "docker-entrypoint.s…"   52 seconds ago   Up 51 seconds       0.0.0.0:6372->6372/tcp, :::6372->6372/tcp, 0.0.0.0:16372->16372/tcp, :::16372->16372/tcp, 6379/tcp   redis-2
    d8fae67a40b2   redis:5.0.14     "docker-entrypoint.s…"   54 seconds ago   Up 52 seconds       0.0.0.0:6371->6371/tcp, :::6371->6371/tcp, 0.0.0.0:16371->16371/tcp, :::16371->16371/tcp, 6379/tcp   redis-1

### 2.2.4 创建集群 ###

# 创建集群
    docker exec -it redis-1 /bin/sh 
    
    # 容器内操作
    cd /usr/local/bin
    redis-cli --cluster create 172.81.0.11:6371 172.81.0.12:6372 172.81.0.13:6373 172.81.0.14:6374 172.81.0.15:6375 172.81.0.16:6376 --cluster-replicas 1
    # 打印出来的信息不再贴出
    
    # 连接集群
    cd /usr/local/redis-5.0.14
    ./bin/redis-cli -p 6371 -c
    cluster nodes
    
    # 退出容器
    exit

### 2.2.5 删除容器及配置文件 ###

# 停止
    for port in $(seq 1 6); \
    do \
    docker stop redis-${port} 
    done
    
    # 删除停止的容器
    docker rm $(docker ps -qa)
    
    # 删除配置及数据文件
    rm -rf ./redis-docker

## 2.3.测试 ##

# 延迟测试
    ./bin/redis-cli -c -p 6371 --intrinsic-latency 10
    # 性能测试
    ./bin/redis-benchmark -h 172.81.0.1 -p 6371  -q

*  延迟测试结果

# 1
    148915461 total runs (avg latency: 0.0672 microseconds / 67.15 nanoseconds per run).
    Worst run took 172533x longer than the average latency.
    # 2
    148930804 total runs (avg latency: 0.0671 microseconds / 67.15 nanoseconds per run).
    Worst run took 147203x longer than the average latency.
    # 3
    149999312 total runs (avg latency: 0.0667 microseconds / 66.67 nanoseconds per run).
    Worst run took 119474x longer than the average latency.

*  性能测试结果

# 1
    PING_INLINE: 49212.60
    PING_BULK: 50327.12
    SET: 48590.86
    GET: 47846.89
    INCR: 48923.68
    LPUSH: 44863.16
    RPUSH: 45106.00
    LPOP: 45599.63
    RPOP: 44682.75
    SADD: 50377.83
    HSET: 48239.27
    SPOP: 50479.56
    LPUSH (needed to benchmark LRANGE): 44345.89
    LRANGE_100 (first 100 elements): 29691.21
    LRANGE_300 (first 300 elements): 14858.84
    LRANGE_500 (first 450 elements): 11119.76
    LRANGE_600 (first 600 elements): 8946.94
    MSET (10 keys): 43859.65
    # 2
    PING_INLINE: 49975.02
    PING_BULK: 51493.30
    SET: 47664.44
    GET: 49358.34
    INCR: 49358.34
    LPUSH: 44822.95
    RPUSH: 45146.73
    LPOP: 44404.97
    RPOP: 45703.84
    SADD: 50125.31
    HSET: 45829.52
    SPOP: 49701.79
    LPUSH (needed to benchmark LRANGE): 42643.92
    LRANGE_100 (first 100 elements): 30349.01
    LRANGE_300 (first 300 elements): 14641.29
    LRANGE_500 (first 450 elements): 11111.11
    LRANGE_600 (first 600 elements): 8964.59
    MSET (10 keys): 44642.86
    # 3
    PING_INLINE: 50075.11
    PING_BULK: 49900.20
    SET: 48971.59
    GET: 49164.21
    INCR: 49925.11
    LPUSH: 44130.62
    RPUSH: 40950.04
    LPOP: 40338.84
    RPOP: 44111.16
    SADD: 48449.61
    HSET: 47687.18
    SPOP: 49455.98
    LPUSH (needed to benchmark LRANGE): 45004.50
    LRANGE_100 (first 100 elements): 30637.26
    LRANGE_300 (first 300 elements): 14788.52
    LRANGE_500 (first 450 elements): 11183.18
    LRANGE_600 (first 600 elements): 9005.76
    MSET (10 keys): 45167.12

![在这里插入图片描述][857b8e411a36448aa5f22eecb9903dd1.png_pic_center]

# 3.总结 #

我们跟之前的单机版本进行对比，延迟对比结果：

![在这里插入图片描述][732889f6d99b45ac86b7dd6b4d501392.png_pic_center]  
性能比对结果：

![在这里插入图片描述][34d609e286b542098c58155a720735b3.png_pic_center]

*  伪集群和在一台服务器上的Docker集群一样无法提升性能。

[27f233a5ce9d4b2392abf4b180c723b1.png_pic_center]: https://img-blog.csdnimg.cn/27f233a5ce9d4b2392abf4b180c723b1.png#pic_center
[857b8e411a36448aa5f22eecb9903dd1.png_pic_center]: https://img-blog.csdnimg.cn/857b8e411a36448aa5f22eecb9903dd1.png#pic_center
[732889f6d99b45ac86b7dd6b4d501392.png_pic_center]: https://img-blog.csdnimg.cn/732889f6d99b45ac86b7dd6b4d501392.png#pic_center
[34d609e286b542098c58155a720735b3.png_pic_center]: https://img-blog.csdnimg.cn/34d609e286b542098c58155a720735b3.png#pic_center