K8S安装RabbitMQ集群及使用

偏执的太偏执、 2022-11-01 13:59 1006阅读 0赞

## 一 实现方案 ##

RabbitMQ Cluster Kubernetes Operator是一个开源的Kubernetes Operator，提供了一种一致且简便的方法，将RabbitMQ集群部署到Kubernetes并运行它们，包括"第二天"(连续）操作，它可以自动的执行Kubernetes上运行的RabbitMQ集群的配置并对集群进行管理和操作。

### 1.1 安装限制 ###

*  Kubernetes 1.16+
 *  RabbitMQ Docker镜像版本 3.8.8+

## 二 实现过程 ##

### 2.1 安装 ###

**（1）项目结构**

├─scripts
      ├─rabbitmq-cluster-operator.yml
      ├─rabbitmq-definition.yaml
    ├─software
      ├─rabbitmq_delayed_message_exchange-3.8.9-0199d11c.ez
    ├─Dockerfile

**（2）Dockerfile**

由于通过rabbitmq-cluster-operator方式安装，没有找到plugins的安装方式，为此通过新建Docker镜像方式安装rabbitmq\_delayed\_message\_exchange插件

FROM rabbitmq:3.8.9-management
    
    MAINTAINER will
    
    ADD rabbitmq_delayed_message_exchange-3.8.9-0199d11c.ez /opt/rabbitmq/plugins/
    
    RUN set -x \
        && chown -R rabbitmq:rabbitmq /opt/rabbitmq/plugins/rabbitmq_delayed_message_exchange-3.8.9-0199d11c.ez \
        && chmod 775 /opt/rabbitmq/plugins/rabbitmq_delayed_message_exchange-3.8.9-0199d11c.ez \
        && rabbitmq-plugins enable rabbitmq_delayed_message_exchange \

**(3) 构建镜像及上传**

docker build -t harbor-sit.wxt.com/public/rabbitmq:3.8.9 . -f Dockerfile
    docker push  harbor-sit.wxt.com/public/rabbitmq:3.8.9

> 注意在k8s的node节点手动pull harbor-sit.wxt.com/public/rabbitmq:3.8.9镜像，否则在kubeclt部署时会出现无法pull镜像问题（harbor其他项目是可正常pull）。

**（4）下载配置文件**

进入github选择k8s匹配版本cluster-operator，并将文件重命名为：`rabbitmq-cluster-operator.yml`

https://github.com/rabbitmq/cluster-operator/tree/0.49.0

> 注意cluster-operator与k8s版本的匹配关系，cluster-operator-0.49.0支持Kubernetes `1.16` or newer

**（5）安装cluster-operator**

可按照我们自己的需求修改相关配置，如namespace等

kubectl apply -f rabbitmq-cluster-operator.yml

输出信息：

customresourcedefinition.apiextensions.k8s.io/rabbitmqclusters.rabbitmq.com created
    serviceaccount/rabbitmq-cluster-operator created
    role.rbac.authorization.k8s.io/rabbitmq-cluster-leader-election-role created
    clusterrole.rbac.authorization.k8s.io/rabbitmq-cluster-operator-role created
    rolebinding.rbac.authorization.k8s.io/rabbitmq-cluster-leader-election-rolebinding created
    clusterrolebinding.rbac.authorization.k8s.io/rabbitmq-cluster-operator-rolebinding created
    deployment.apps/rabbitmq-cluster-operator created

**（4）配置MQ实例**

新建 `rabbitmq-definition.yaml`文件，进行如下配置：

apiVersion: rabbitmq.com/v1beta1
    kind: RabbitmqCluster
    metadata:
      name: rabbitmq #服务名称
      namespace: yrz-sit-software
    spec:
      image: harbor-sit.wxt.com/public/rabbitmq:3.8.9
      replicas: 3 #副本数
      service:
        type: NodePort # 服务类型
      persistence:
        storageClassName: nfs-client #存储类的名称
        storage: 5Gi
      resources:
        requests:
          cpu: 200m
          memory: 400Mi
        limits:
          cpu: 300m
          memory: 600Mi
      rabbitmq:
        additionalPlugins:
          - rabbitmq_delayed_message_exchange   # 启用插件

> 注意：resources根据具体环境配置

**(5) 部署MQ实例**

kubectl apply -f rabbitmq-definition.yaml

获取：

kubectl get all -l app.kubernetes.io/name=rabbitmq -n yrz-sit-software

结果：

NAME                    READY   STATUS    RESTARTS   AGE
    pod/rabbitmq-server-0   1/1     Running   0          112s
    pod/rabbitmq-server-1   1/1     Running   0          80s
    pod/rabbitmq-server-2   1/1     Running   0          54s
    
    NAME                     TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                          AGE
    service/rabbitmq         NodePort    10.68.125.158   <none>        5672:36939/TCP,15672:29706/TCP   112s
    service/rabbitmq-nodes   ClusterIP   None            <none>        4369/TCP,25672/TCP               112s
    
    NAME                               READY   AGE
    statefulset.apps/rabbitmq-server   3/3     112s

**（6）修改svc默认端口**

根据实际情况决定是否修改， nodePort: 36939修改为nodePort: 20749，nodePort: 29706修改为nodePort: 20057

kubectl edit svc/rabbitmq -n yrz-sit-software -o yaml --save-config
    
    spec:
      clusterIP: 10.68.125.158
      externalTrafficPolicy: Cluster
      ports:
      - name: amqp
        nodePort: 20749
        port: 5672
        protocol: TCP
        targetPort: 5672
      - name: management
        nodePort: 20057
        port: 15672
        protocol: TCP
        targetPort: 15672
      selector:
        app.kubernetes.io/name: rabbitmq
      sessionAffinity: None
      type: NodePort

**（7）获取账号和密码**

kubectl -n yrz-sit-software get secret rabbitmq-default-user  -o jsonpath="{.data.username}" | base64 --decode
    kubectl -n yrz-sit-software get secret rabbitmq-default-user -o jsonpath="{.data.password}" | base64 --decode

## 三 集群健康检查 ##

# 1. rabbitmq访问路径
    http://10.168.xx.xx:20057
    
    # 用户名/密码
    Gfz7siXNrPBCu24T_bCQeOhp2TZWqKk3/6itjHADQz3D0SvVOVhELcJLhGgYZjSaw
    
    # 2 获取域名 rabbitmq StatefulSet
    [root@yrz-k8s-master01 rabbitmq]# for i in 0 1 2; do kubectl exec -n yrz-sit-software rabbitmq-server-$i -- hostname -f; done
    
    rabbitmq-server-0.rabbitmq-nodes.yrz-sit-software.svc.cluster.local.
    rabbitmq-server-1.rabbitmq-nodes.yrz-sit-software.svc.cluster.local.
    rabbitmq-server-2.rabbitmq-nodes.yrz-sit-software.svc.cluster.local.

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhbmdjaGVuZ2FpaHVpbWluZw_size_16_color_FFFFFF_t_70_pic_center]

## 四 问题记录 ##

# 出现无法删除crd
    kubectl delete crd rabbitmqclusters.rabbitmq.com
    
    # 解决
    kubectl patch crd/rabbitmqclusters.rabbitmq.com -p '{"metadata":{"finalizers":[]}}' --type=merge

--------------------

欢迎关注公众号： **编码是个技术活**

![在这里插入图片描述][20210219162337578.jpg_pic_center]

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dhbmdjaGVuZ2FpaHVpbWluZw_size_16_color_FFFFFF_t_70_pic_center]: /images/20221024/5998b9f6195a4cb6b7d7e670f5fa0160.png
[20210219162337578.jpg_pic_center]: /images/20221024/e1f4c357a35f4e50a9074d9ed6a5b545.png