使用docker部署redis-sentinel 和redis-cluster

亦凉 2022-03-15 03:40 358阅读 0赞 
  1.    写这篇文章的目的并不是为了介绍redis以及它的高可用方案,主要是通过redis各种高可用方案的简单安装,让读者能够通过真实的集群环境更好的体会redis的高可用方案。对redis感兴趣的同学经常会遇到这样的一个问题，没有足够的虚拟机,因此有时候想要自己部署redis各种集群环境时总是显得力不从心,而使用官方的安装包又非常麻烦,因此本文利用docker-compose来快速安装redis，当然，前提是你要会简单使用docker和docker-compose哦。
  3.     话不多说，咱们直接进入正题,下面所有的操作都是在我的一台虚拟机(192.168.174.129)上完成的,读者在使用时可以把配置中涉及到的ip改成自己的实际ip.

一、单实例的redis

创建 /home/ubuntu/redis/standalone/docker-compose.yml，如下

  1. version: '2'
  2. services:
  3.   redis:
  4.     image: redis:5.0-rc-alpine
  5.     restart: always
  6.     volumes:
  7.       - "/etc/timezone:/etc/timezone:ro"
  8.       - "/etc/localtime:/etc/localtime:ro"
  9.     ports:
  10.       - 6379:6379

$: cd /home/ubuntu/redis/standalone/

$: docker-compose up -d

然后,一个redis实例就启动好了,就是这么easy。此时只需要使用你的redis client访问192.168.174.129的6379端口就可以愉快的使用redis了，可以简单的验证一下,如下,还是在/home/ubuntu/redis/standalone目录下执行如下命令

$: docker-compose exec redis redis-cli

进入redis-cli的命令行，然后就可以在里面验证各种redis命令了

20190228182423551.png

二、redis一主多从(读写分离)

  1.  创建  /home/ubuntu/redis/repl/docker-compose.yml，如下
  3. version: '2'
  4. services:
  5.   master:
  6.     image: redis:5.0-rc-alpine
  7.     restart: always
  8.     volumes:
  9.       - "/etc/timezone:/etc/timezone:ro"
  10.       - "/etc/localtime:/etc/localtime:ro"
  11.     ports:
  12.       - 6380:6379
  14.   slave-1:
  15.     image: redis:5.0-rc-alpine
  16.     restart: always
  17.     volumes:
  18.       - "/etc/timezone:/etc/timezone:ro"
  19.       - "/etc/localtime:/etc/localtime:ro"
  20.     ports:
  21.       - 6381:6379
  22.     command: ["redis-server","--slaveof","192.168.174.129","6380"]
  24.   slave-2:
  25.     image: redis:5.0-rc-alpine
  26.     restart: always
  27.     volumes:
  28.       - "/etc/timezone:/etc/timezone:ro"
  29.       - "/etc/localtime:/etc/localtime:ro"
  30.     ports:
  31.       - 6382:6379
  32.     command: ["redis-server","--slaveof","192.168.174.129","6380"]
  34.   slave-3:
  35.     image: redis:5.0-rc-alpine
  36.     restart: always
  37.     volumes:
  38.       - "/etc/timezone:/etc/timezone:ro"
  39.       - "/etc/localtime:/etc/localtime:ro"
  40.     ports:
  41.       - 6383:6379
  42.     command: ["redis-server","--slaveof","192.168.174.129","6382"]

注：slave-3是slave-2的从服务器,slave-2和slave-1是master的从服务器,这样举例主要是为了体现嵌套树状复制结构 。

三、redis-sentinel

创建 /home/ubuntu/redis/sentinel/docker-compose.yml，如下

  1. version: '2'
  2. services:
  3.   repl-0:
  4.     image: redis:5.0-rc-alpine
  5.     restart: always
  6.     volumes:
  7.       - "/etc/timezone:/etc/timezone:ro"
  8.       - "/etc/localtime:/etc/localtime:ro"
  9.     ports:
  10.       - 6380:6379       
  11.     networks:
  12.       - sentinel
  13.     command: ["redis-server","--replica-announce-ip","192.168.174.129","--replica-announce-port","6380"]
  15.   repl-1:
  16.     image: redis:5.0-rc-alpine
  17.     restart: always
  18.     volumes:
  19.       - "/etc/timezone:/etc/timezone:ro"
  20.       - "/etc/localtime:/etc/localtime:ro"
  21.     ports:
  22.       - 6381:6379       
  23.     networks:
  24.       - sentinel
  25.     command: ["redis-server","--slaveof","repl-0","6379","--replica-announce-ip","192.168.174.129","--replica-announce-port","6381"]
  27.   repl-2:
  28.     image: redis:5.0-rc-alpine
  29.     restart: always
  30.     volumes:
  31.       - "/etc/timezone:/etc/timezone:ro"
  32.       - "/etc/localtime:/etc/localtime:ro"
  33.     ports:
  34.       - 6382:6379       
  35.     networks:
  36.       - sentinel
  37.     command: ["redis-server","--slaveof","repl-0","6379","--replica-announce-ip","192.168.174.129","--replica-announce-port","6382"]
  39.   repl-3:
  40.     image: redis:5.0-rc-alpine
  41.     restart: always
  42.     volumes:
  43.       - "/etc/timezone:/etc/timezone:ro"
  44.       - "/etc/localtime:/etc/localtime:ro"
  45.     ports:
  46.       - 6383:6379       
  47.     networks:
  48.       - sentinel
  49.     ports:
  50.       - 6383:6379
  51.     command: ["redis-server","--slaveof","repl-2","6379","--replica-announce-ip","192.168.174.129","--replica-announce-port","6383"]
  53.   sentinel-1:
  54.     build: ./build
  55.     restart: always
  56.     volumes:
  57.       - "/etc/timezone:/etc/timezone:ro"
  58.       - "/etc/localtime:/etc/localtime:ro"
  59.     networks:
  60.       - sentinel 
  61.     ports:
  62.       - 26380:26379
  63.     command: ["redis-server","/data/sentinel.conf","--sentinel","announce-ip","192.168.174.129","--sentinel","announce-port" , "26380"]
  65.   sentinel-2:
  66.     build: ./build
  67.     restart: always
  68.     volumes:
  69.       - "/etc/timezone:/etc/timezone:ro"
  70.       - "/etc/localtime:/etc/localtime:ro"
  71.     networks:
  72.       - sentinel 
  73.     ports:
  74.       - 26381:26379
  75.     command: ["redis-server","/data/sentinel.conf","--sentinel","announce-ip","192.168.174.129","--sentinel","announce-port" , "26381"]
  77.   sentinel-3:
  78.     build: ./build
  79.     restart: always
  80.     volumes:
  81.       - "/etc/timezone:/etc/timezone:ro"
  82.       - "/etc/localtime:/etc/localtime:ro"
  83.     networks:
  84.       - sentinel  
  85.     ports:
  86.       - 26382:26379
  87.     command: ["redis-server","/data/sentinel.conf","--sentinel","announce-ip","192.168.174.129","--sentinel","announce-port" , "26382"]
  89. networks:
  90.   sentinel: 
  91.      driver: bridge

注：与非docker环境相比,我们添加了announce-ip和announce-port这些命令,这是因为默认情况下主从服务器之间以及各个sentinel实例之间使用的ip是容器内部自动分配的ip,这样在切换主从服务或sentinel之间通信时无法联通，因此务必需要添加对应的命令。

同时在/home/ubuntu/redis/sentinel/build目录下创建 Dockerfile和sentinel.conf

Dockerfile:

  1. FROM redis:5.0-rc-alpine
  2. COPY ./sentinel.conf /data/sentinel.conf
  3. EXPOSE 26379

sentinel.conf

  1. sentinel monitor mymaster 192.168.174.129 6380 2
  2. sentinel down-after-milliseconds mymaster 10000
  3. sentinel failover-timeout mymaster 180000
  4. sentinel parallel-syncs mymaster 1

四、redis-cluster

创建 /home/ubuntu/redis/cluster/docker-compose.yml，如下

  1. version: '2'
  2. services:
  3.   cluster-a1:
  4.     image: redis:5.0-rc-alpine
  5.     restart: always
  6.     volumes:
  7.       - "/etc/timezone:/etc/timezone:ro"
  8.       - "/etc/localtime:/etc/localtime:ro"
  9.       - "./conf.d/cluster-6371.conf:/data/redis.conf" 
  10.     network_mode: host
  11.     command: ["redis-server","/data/redis.conf"]
  13.   cluster-a2:
  14.     image: redis:5.0-rc-alpine
  15.     restart: always
  16.     volumes:
  17.       - "/etc/timezone:/etc/timezone:ro"
  18.       - "/etc/localtime:/etc/localtime:ro"
  19.       - "./conf.d/cluster-6372.conf:/data/redis.conf" 
  20.     network_mode: host
  21.     command: ["redis-server","/data/redis.conf"]
  23.   cluster-b1:
  24.     image: redis:5.0-rc-alpine
  25.     restart: always
  26.     volumes:
  27.       - "/etc/timezone:/etc/timezone:ro"
  28.       - "/etc/localtime:/etc/localtime:ro"
  29.       - "./conf.d/cluster-6381.conf:/data/redis.conf" 
  30.     network_mode: host
  31.     command: ["redis-server","/data/redis.conf"]
  33.   cluster-b2:
  34.     image: redis:5.0-rc-alpine
  35.     restart: always
  36.     volumes:
  37.       - "/etc/timezone:/etc/timezone:ro"
  38.       - "/etc/localtime:/etc/localtime:ro"
  39.       - "./conf.d/cluster-6382.conf:/data/redis.conf"
  40.     network_mode: host
  41.     command: ["redis-server","/data/redis.conf"]
  43.   cluster-c1:
  44.     image: redis:5.0-rc-alpine
  45.     restart: always
  46.     volumes:
  47.       - "/etc/timezone:/etc/timezone:ro"
  48.       - "/etc/localtime:/etc/localtime:ro"
  49.       - "./conf.d/cluster-6391.conf:/data/redis.conf" 
  50.     network_mode: host
  51.     command: ["redis-server","/data/redis.conf"]
  53.   cluster-c2:
  54.     image: redis:5.0-rc-alpine
  55.     restart: always
  56.     volumes:
  57.       - "/etc/timezone:/etc/timezone:ro"
  58.       - "/etc/localtime:/etc/localtime:ro"
  59.       - "./conf.d/cluster-6392.conf:/data/redis.conf"
  60.     network_mode: host
  61.     command: ["redis-server","/data/redis.conf"]

同时在 /home/ubuntu/redis/cluster/conf.d目录下创建cluster-6371.conf 、 cluster-6372.conf 、cluster-6381.conf 、cluster-6382.conf 、cluster-6391.conf 、cluster-6392.conf ，这六个配置文件唯一的区别只是第一行的port不一样,每个配置文件的port与文件名上的数字一致,如下面是cluster-6371.conf

  1. port 6371
  2. cluster-enabled yes
  3. cluster-node-timeout 15000
  4. cluster-config-file "nodes.conf"
  5. pidfile /var/run/redis.pid
  6. logfile "cluster.log"
  7. dbfilename dump-cluster.rdb
  8. appendfilename "appendonly-cluster.aof"

接下来,在/home/ubuntu/redis/cluster目录下进行如下操作启动所有redis实例

$: docker-compose up -d

-- 查看容器的状态

$: docker-compose ps

20190301145407908.png

(1) 将所有实例加入集群

$: docker-compose exec cluster-c1 redis-cli -p 6381 -h 192.168.174.129

进入redis客户端命令行

192.168.174.129:6381> cluster meet 192.168.174.129 6371

192.168.174.129:6381> cluster meet 192.168.174.129 6372

192.168.174.129:6381> cluster meet 192.168.174.129 6381

192.168.174.129:6381> cluster meet 192.168.174.129 6382

192.168.174.129:6381> cluster meet 192.168.174.129 6391

192.168.174.129:6381> cluster meet 192.168.174.129 6392

检查一下效果

192.168.174.129:6381> cluster nodes

20190301114054136.png

注1：上面的第一列是node_id，这个在步骤(3)中会用到,比如16a7e8a557ba01615c5b15e967c769f983913b85是192.168.174.129:6392的node_id。

注2：当前的6个node都是master，还没有分配主从关系，具体见步骤(3)

（2）分配虚拟槽

  1. 可以看到配置的六个节点都已经加入到了集群中，但是其现在还不能使用，因为还没有将16384个槽分配到集群节点中。虚拟槽的分配可以使用redis-cli分别连接到6371，6381和6391端口的节点中，然后分别执行如下命令：

$: docker-compose exec cluster-c1 redis-cli -p 6371 -h 192.168.174.129 cluster addslots {0..5000}

$: docker-compose exec cluster-c1 redis-cli -p 6381 -h 192.168.174.129 cluster addslots {5001..10000}

$: docker-compose exec cluster-c1 redis-cli -p 6391 -h 192.168.174.129 cluster addslots {10001..16383}

-- 检查当前集群的状态

$: docker-compose exec cluster-c1 redis-cli -p 6381 -h 192.168.174.129 cluster info

watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NvbmdfamF2YQ_size_16_color_FFFFFF_t_70

（3）设置主从关系

  1.  通过上面的步骤，我们把16384个槽分配给了6371、6381、6391这三个端口对应的redis实例,因此这三个实例必须是master，至少为每一个master再分配一个slave，如下命令 : cluster replicate \[master\_node\_id\],

-- 将6372对应的实例设置为6371的slave

$: docker-compose exec cluster-c1 redis-cli -p 6372 -h 192.168.174.129 cluster replicate 16a7e8a557ba01615c5b15e967c769f983913b85

-- 将6382对应的实例设置为6381的slave

$: docker-compose exec cluster-c1 redis-cli -p 6382 -h 192.168.174.129 cluster replicate 3921bc89204a900360de770fe42f931787431e7c

-- 将6392对应的实例设置为6391的slave

$: docker-compose exec cluster-c1 redis-cli -p 6392 -h 192.168.174.129 cluster replicate fad1e9b5115c876dbbaf3ba4c500ddfca0508065

-- 检查当前的节点主从关系

$: docker-compose exec cluster-c1 redis-cli -p 6382 -h 192.168.174.129 cluster nodes

20190301145138351.png

（4）使用redis-cli连接集群

$: docker-compose exec cluster-c1 redis-cli -p 6382 -h 192.168.174.129 -c

进入redis-cli控制台

192.168.174.129:6382> get k1

20190301150123920.png

发现跳转到6391对应的实例上,你也可以试着停掉其中的一个master，然后观察一下他的slave是否会自动转为master。

还有更多redis-cluster的特性，都可以在这个集群上进行测试验证，小伙伴们还等什么，赶紧动手吧。

注： 细心的同学会发现，我们使用redis-cli时都是使用了”docker-compose exec cluster-c1 redis-cli “，其实我们也可以使用其他的redis实例,比如 “docker-compose exec cluster-a2 redis-cli “,我们只是为了使用redis实例中内置的redis-cli命令,如果你本地已经装了redis-cli,也是可以直接使用redis-cli。

五、总结

  1.   关于redis主从复制、sentinel高可用以及redis-cluster，网上有很多优秀的文章对他们进行了非常详细的介绍，我在这里不在赘述，只是简单说一下我个人不太成熟的理解吧。redis主从复制可以实现读写分离,但是在master挂掉之后没有现成的机制重新选举，redis的写功能就不可用了，所以引出了sentinel，sentinel可以在master挂掉之后可以选一个slave替代挂掉的master，所以sentinel增强了高可用性,但是没有解决redis水平扩展的问题,当master的内存不足时，除了增加机器内存,没有现成的办法扩容，因此引入了redis-cluster，cluster通过哈希槽让各个master节点只负责一部分的哈希槽（hash slo)，然后每个master有多个slave作为数据备份,一旦master不行了，可以让slave顶上去。
  3.   sentinel和cluster都不能很好的使用读写分离,slave节点只是作为备份节点，因此slave节点不能替master分担读的压力，同时由于redis使用的是异步复制,不能保证强一致性。
  5.   有一些不错的帖子,大家可以参考一些

https://my.oschina.net/zhangxufeng/blog/905611

https://dbaplus.cn/news-158-2182-1.html

redis镜像地址： https://hub.docker.com/_/redis

官方地址: https://redis.io/

英文不好的同学可以参考 http://www.redis.cn/

docker安装请参考 https://blog.csdn.net/song_java/article/details/88061162

发表评论

表情：
评论列表 （有 0 条评论，358人围观）

还没有评论，来说两句吧...

相关阅读