Kubernetes入门（六）

﹏ヽ暗。殇╰゛Y 2022-03-17 06:16 193阅读 0赞

## 30.9 Kubernetes中使用NFS作为共享存储 ##

--------------------

前面介绍过，Persistent Volume（持久化卷）简称PV，是一个k8s资源对象。我们可以单独创建一个PV，它不和Pod直接相关，而是通过Persistent Volume Claim（简称PVC）来实现动态绑定，我们会在Pod定义里指定创建好的PVC，然后PVC会根据Pod的要求去自动绑定合适的PV给Pod使用。

--------------------

### PV和PVC概念 ###

Persistent Volume（PV）是由管理员设置的存储，它是集群的一部分，就像节点是集群中的资源一样，PV也是集群中的资源。PV是Volume之类的卷插件，但具有独立于使用PV的Pod的生命周期。此API对象包含存储实现的细节，即NFS、iSCSI或特定于云供应商的存储系统。

Persistent Volume Claim（PVC）是用户存储的请求。它与Pod相似，Pod消耗节点资源，PVC消耗PV资源。Pod可以请求特定级别的资源（CPU和内存），PVC声明可以请求特定的大小和访问模式。

普通Volume和使用它的Pod之间是一种静态绑定关系，在定义Pod的文件里，同时定义了它使用的Volume。Volume是Pod的附属品，我们无法单独创建一个Volume，因为它不是一个独立的k8s资源对象。

--------------------

### 搭建NFS ###

接下来还是在master2节点上搭建NFS。

*  安装包：

[root@master2 ~]# yum install -y nfs-utils rpcbind

*  编辑配置文件：

[root@master2 ~]# vim /etc/exports
    /data/k8s/ 192.168.30.0/24(sync,rw,no_root_squash)
    
    [root@master2 ~]# mkdir /data/k8s
    
    [root@master2 ~]# chmod 777 !$

*  启动服务：

[root@master2 ~]# systemctl start rpcbind; systemctl start nfs
    
    [root@master2 ~]# systemctl enable rpcbind; systemctl enable nfs

到node2节点上查看

[root@node2 ~]# yum install -y nfs-utils
    
    [root@node2 ~]# showmount -e 192.168.30.150
    Export list for 192.168.30.150:
    /data/k8s 192.168.30.0/24

--------------------

### 创建PV ###

*  master上创建PV：

[root@master ~]# vim mypv.yaml

apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: pv001
    spec:
      capacity:
        storage: 10Gi
      accessModes:
        - ReadWriteMany                 #读写权限，允许被多个Node挂载
      nfs:
        path: /data/k8s
        server: 192.168.30.150              #指定nfs路径

[root@master ~]# kubectl get pv
    NAME      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM     STORAGECLASS   REASON    AGE
    pv001     10Gi       RWX            Retain           Available                                      37s

*  创建PVC：

[root@master ~]# vim mypvc.yaml

apiVersion: v1
    kind: PersistentVolumeClaim
    apiVersion: v1
    metadata:
      name: myclaim
    spec:
      accessModes:
        - ReadWriteMany
      resources:
        requests:
          storage: 8Gi

[root@master ~]# kubectl create -f mypvc.yaml
    
    [root@master ~]# kubectl get pvc
    NAME      STATUS    VOLUME    CAPACITY   ACCESS MODES   STORAGECLASS   AGE
    myclaim   Bound     pv001     10Gi       RWX                           32s                  #自动绑定pv001

*  定义Pod：

[root@master ~]# vim pv-pod.yaml

apiVersion: v1
    kind: Pod
    metadata:
      name: httpd-pvpod
    spec:
      containers:
      - image: httpd
        name: httpd-withpvc-pod
        imagePullPolicy: Always                     #获取镜像的策略 Alawys表示下载镜像
        volumeMounts:                           #挂载到容器内部的存储卷配置
        - mountPath: "/usr/local/apache2/htdocs/"                   ##存储卷在容器内mount的绝对路径
          name: httpd-volume
      volumes:
        - name: httpd-volume
          persistentVolumeClaim:
            claimName: myclaim

[root@master ~]# kubectl create -f pv-pod.yaml
    
    [root@master ~]# kubectl get pod
    NAME                     READY     STATUS    RESTARTS   AGE
    httpd-pod                1/1       Running   0          1h
    httpd-pvpod              1/1       Running   0          22s

*  验证：

到NFS的共享目录下创建一个文件

[root@master2 ~]# cd /data/k8s/
    
    [root@master2 k8s]# echo "just test" > 1.html

进入到httpd-pvpod里面

[root@master ~]# kubectl exec -it httpd-pvpod bash
    root@httpd-pvpod:/usr/local/apache2# ls htdocs/
    1.html
    root@httpd-pvpod:/usr/local/apache2# cat htdocs/1.html 
    just test                   #与上面是一致的

删除pod httpd-pvpod

[root@master ~]# kubectl delete pod httpd-pvpod 
    pod "httpd-pvpod" deleted
    
    [root@master2 k8s]# cat /data/k8s/1.html 
    just test

重建pod httpd-pvpod

[root@master ~]# kubectl create -f pv-pod.yaml

curl访问

[root@master ~]# kubectl get pod httpd-pvpod -o wide #查看pod ip
    NAME          READY     STATUS    RESTARTS   AGE       IP            NODE            NOMINATED NODE
    httpd-pvpod   1/1       Running   0          5m        172.20.2.13   192.168.1.253   <none>
    
    [root@master ~]# curl 172.20.2.13/1.html
    just test

这就说明PV存储的数据是永久的，和Pod不相关，Pod的删除、重建不会影响到PV中的数据。

--------------------

## 30.10 在Kubernetes中部署LNMP并运行Discuz ##

--------------------

在部署了kubernetes集群和harbor之后，我们开始部署LNMP环境，并运行Discuz。

--------------------

### 准备镜像 ###

*  下载镜像：

[root@master ~]# docker pull mysql:5.6
    
    [root@master ~]# docker pull richarvey/nginx-php-fpm

*  用dockerfile重建nginx-php-fpm镜像：

Dockerfile  localtime  nginx.conf  php-fpm-www.conf 应该在同一目录下，这里省略localtime  nginx.conf  php-fpm-www.conf，请自定义配置
    
    [root@master ~]# vim Dockerfile
    FROM richarvey/nginx-php-fpm
    COPY localtime /etc/localtime
    RUN echo 'Asia/Shanghai' >/etc/timezone
    RUN rm -f /etc/nginx/nginx.conf
    COPY nginx.conf /etc/nginx/nginx.conf
    RUN rm -f /usr/local/etc/php-fpm.d/www.conf
    COPY php-fpm-www.conf /usr/local/etc/php-fpm.d/www.conf
    ENTRYPOINT /usr/sbin/nginx -c /etc/nginx/nginx.conf && /usr/local/sbin/php-fpm -c /usr/local/etc/php-fpm.conf
    
    [root@master ~]# docker build -t nginx.php .

*  推送镜像到harbor上：

[root@master ~]# docker login harbor.lzxlinux.com
    
    [root@master ~]# docker tag nginx.php harbor.lzxlinux.com/lzx/nginx-php
    
    [root@master ~]# docker push harbor.lzxlinux.com/lzx/nginx-php
    
    [root@master ~]# docker tag mysql:5.6 harbor.lzxlinux.com/lzx/mysql:5.6
    
    [root@master ~]# docker push harbor.lzxlinux.com/lzx/mysql:5.6

刷新页面查看

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70]

--------------------

### 搭建NFS ###

NFS上面已经搭建好了，这里省略，如果没有搭建NFS请先搭建。

*  创建目录：

[root@master2 ~]# mkdir -p /data/k8s/discuz/{db,web}

--------------------

### 搭建MySQL服务 ###

*  创建secret（设定mysql的root密码）：

[root@master ~]# kubectl create secret generic mysql-pass --from-literal=password=DzPasswd1

*  创建PV：

[root@master ~]# vim mysql-pv.yaml

apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: mysql-pv
    spec:
      capacity:
        storage: 10Gi
      accessModes:
        - ReadWriteMany
      nfs:
        path: /data/k8s/discuz/db
        server: 192.168.30.150

[root@master ~]# kubectl create -f mysql-pv.yaml

*  创建PVC：

[root@master ~]# vim mysql-pvc.yaml

apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: mysql-claim
      labels:
        app: discuz
    spec:
      accessModes:
        - ReadWriteMany
      resources:
        requests:
          storage: 10Gi

[root@master ~]# kubectl create -f mysql-pvc.yaml
    
    [root@master ~]# kubectl get pv
    NAME       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM                 STORAGECLASS   REASON    AGE
    mysql-pv   10Gi       RWX            Retain           Bound     default/mysql-claim                            50s
    
    [root@master ~]# kubectl get pvc
    NAME          STATUS    VOLUME     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
    mysql-claim   Bound     mysql-pv   10Gi       RWX                           57s                     #自动绑定mysql-pv

*  创建Deployment：

[root@master ~]# vim mysql-deploy.yaml

apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: dz-mysql
      labels:
        app: discuz
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: discuz
          tier: mysql
      strategy:
        type: Recreate
      template:
        metadata:
          labels:
            app: discuz
            tier: mysql
        spec:
          imagePullSecrets:
          - name: my-secret
          containers:
          - image: harbor.lzxlinux.com/lzx/mysql:5.6
            name: mysql
            env:
            - name: MYSQL_ROOT_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: mysql-pass
                  key: password
            ports:
            - containerPort: 3306
              name: dz-mysql
            volumeMounts:
            - name: mysql-persistent-storage
              mountPath: /var/lib/mysql
          volumes:
          - name: mysql-persistent-storage
            persistentVolumeClaim:
              claimName: mysql-claim

[root@master ~]# kubectl create -f mysql-deploy.yaml
              
    [root@master ~]# kubectl get deploy
    NAME       DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
    dz-mysql   1         1         1            1           43s

*  创建Service：

[root@master ~]# vim mysql-svc.yaml

apiVersion: v1
    kind: Service
    metadata:
      name: dz-mysql
      labels:
        app: discuz
    spec:
      ports:
        - port: 3306
      selector:
        app: discuz
        tier: mysql

[root@master ~]# kubectl create -f mysql-svc.yaml 
    
    [root@master ~]# kubectl get svc
    NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
    dz-mysql     ClusterIP   10.68.3.199     <none>        3306/TCP   1m
    kubernetes   ClusterIP   10.68.0.1       <none>        443/TCP    2d

安装mysql测试一下服务是否正常

[root@master ~]# yum install -y mysql
    
    [root@master ~]# mysql -uroot -pDzPasswd1 -h10.68.3.199 #可以连上去说明没问题
    
    MySQL [(none)]> show databases;                 
    +--------------------+
    | Database           |
    +--------------------+
    | information_schema |
    | mysql              |
    | performance_schema |
    +--------------------+
    3 rows in set (0.00 sec)

--------------------

### 搭建Nginx+php-fpm服务 ###

*  创建PV：

[root@master ~]# vim web-pv.yaml

apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: web-pv
    spec:
      capacity:
        storage: 10Gi
      accessModes:
        - ReadWriteMany
      nfs:
        path: /data/k8s/discuz/web
        server: 192.168.30.150

[root@master ~]# kubectl create -f web-pv.yaml

*  创建PVC：

[root@master ~]# vim web-pvc.yaml

apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: web-claim
      labels:
        app: discuz
    spec:
      accessModes:
        - ReadWriteMany
      resources:
        requests:
          storage: 10Gi

[root@master ~]# kubectl create -f web-pvc.yaml
    
    [root@master ~]# kubectl get pv
    NAME       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM                 STORAGECLASS   REASON    AGE
    mysql-pv   10Gi       RWX            Retain           Bound     default/mysql-claim                            29m
    web-pv     10Gi       RWX            Retain           Bound     default/web-claim                              5m
    
    [root@master ~]# kubectl get pvc
    NAME          STATUS    VOLUME     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
    mysql-claim   Bound     mysql-pv   10Gi       RWX                           30m
    web-claim     Bound     web-pv     10Gi       RWX                           13s                 #自动绑定web-pv

*  创建Deployment：

[root@master ~]# vim mysql-deploy.yaml

apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: dz-web
      labels:
        app: discuz
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: discuz
          tier: nginx-php
      template:
        metadata:
          labels:
            app: discuz
            tier: nginx-php
        spec:
          imagePullSecrets:
          - name: my-secret
          containers:
          - image: harbor.lzxlinux.com/lzx/nginx-php
            name: dz-web
            ports:
            - containerPort: 9000
            - containerPort: 80
              name: dz-web
            volumeMounts:
            - name: mysql-persistent-storage
              mountPath: /var/www/html/
          volumes:
          - name: mysql-persistent-storage
            persistentVolumeClaim:
              claimName: web-claim

[root@master ~]# kubectl create -f web-deploy.yaml
    
    [root@master ~]# kubectl get deploy
    NAME       DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
    dz-mysql   1         1         1            1           43m
    dz-web     1         1         1            1           37s

*  创建Service：

[root@master ~]# vim web-svc.yaml

apiVersion: v1
    kind: Service
    metadata:
      name: dz-web
      labels:
        app: discuz
    spec:
      type: NodePort
      ports:
        - port: 80
          nodePort : 30001
      selector:
        app: discuz
        tier: nginx-php

[root@master ~]# kubectl create -f web-svc.yaml 
    
    [root@master ~]# kubectl get svc
    NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
    dz-mysql     ClusterIP   10.68.3.199     <none>        3306/TCP       1h
    dz-web       NodePort    10.68.137.104   <none>        80:30001/TCP   23s
    kubernetes   ClusterIP   10.68.0.1       <none>        443/TCP        4h

[root@master ~]# curl 10.68.137.104
    <html>
    <head><title>403 Forbidden</title></head>
    <body bgcolor="white">
    <center><h1>403 Forbidden</h1></center>
    <hr><center>nginx/1.14.0</center>
    </body>
    </html>

master2节点上测试一下

[root@master2 ~]# echo "111" > /data/k8s/discuz/web/index.html
    
    [root@master2 ~]# curl 10.68.137.104
    111
    
    [root@master2 ~]# vim /data/k8s/discuz/web/1.php
    <?php
    phpinfo();
    ?>
    
    [root@master2 ~]# curl 10.68.137.104/1.php
    .
    .                   #省略
    .
    This program is free software; you can redistribute it and/or modify it under the terms of the PHP License as published by the PHP Group and included in the distribution in the file:  LICENSE
    </p>
    <p>This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
    </p>
    <p>If you did not receive a copy of the PHP license, or have any questions about PHP licensing, please contact license@php.net.
    </p>
    </td></tr>
    </table>                    # php解析没问题

--------------------

### 安装Discuz ###

*  下载代码：

[root@master2 ~]# cd /tmp/
    
    [root@master2 tmp]# git clone https://gitee.com/ComsenzDiscuz/DiscuzX.git
    
    [root@master2 tmp]# cd /data/k8s/discuz/web/
    
    [root@master2 web]# mv /tmp/DiscuzX/upload/* .
    
    [root@master2 web]# kubectl exec dz-web-7f669d4dd5-8jtrs id nginx
    uid=100(nginx) gid=101(nginx) groups=101(nginx),101(nginx)                  #容器内nginx UID=100
    
    [root@master2 web]# chown -R 100 data uc_server/data/ uc_client/data/ config/

*  设置MySQL普通用户：

[root@master2 web]# kubectl get svc dz-mysql 
    NAME       TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)    AGE
    dz-mysql   ClusterIP   10.68.3.199   <none>        3306/TCP   1h
    
    [root@master2 web]# yum install -y mysql
    
    [root@master2 web]# mysql -uroot -pDzPasswd1 -h10.68.3.199
    
    MySQL [(none)]> create database dz;
    Query OK, 1 row affected (0.01 sec)
    
    MySQL [(none)]> grant all on dz.* to 'dz'@'%' identified by 'dz-passwd-lzx';            #创建用户设置密码
    Query OK, 0 rows affected (0.01 sec)

*  设置nginx代理：

在master节点上做个nginx反向代理

[root@master ~]# kubectl get pod -o wide
    NAME                       READY     STATUS    RESTARTS   AGE       IP           NODE            NOMINATED NODE
    dz-mysql-5fdd755df-swt27   1/1       Running   0          1h        172.20.3.6   192.168.30.130   <none>
    dz-web-7f669d4dd5-8jtrs    1/1       Running   0          47m       172.20.2.6   192.168.30.130   <none>

[root@master ~]# yum install -y nginx
    
    [root@master ~]# vim /etc/nginx/nginx.conf
        server { 
            listen       80 ;
            server_name  dz.lzxlinux.com;
    
            location / { 
                  proxy_pass      http://10.68.137.104:80;              #此处IP为dz-web的cluster ip
                  proxy_set_header Host   $host;
                  proxy_set_header X-Real-IP      $remote_addr;
                  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            }
        }
    
    [root@master ~]# systemctl start nginx

编辑Windows上的hosts文件，路径：`C:\Windows\System32\drivers\etc\hosts`，增加一行：192.168.30.128 [dz.lzxlinux.com][] 。

*  安装Discuz：

设置完hosts后，直接浏览器访问，按步骤设置完成安装。

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 1]

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 2]

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 3]

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 4]

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 5]

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 6]

至此，用k8s部署Discuz完成。

--------------------

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70]: /images/20220317/99f3849182d54cb7a721c21d5845bda3.png
[dz.lzxlinux.com]: http://dz.lzxlinux.com
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 1]: /images/20220317/2d61ed62574648a3bdbb5e3756bcd887.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 2]: /images/20220317/fae28bbc047e4b53820c57e42a0c5cbe.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 3]: /images/20220317/8d4a985e959e47298c7329772679994b.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 4]: /images/20220317/f9510097e45547538254c5c197928713.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 5]: /images/20220317/08efcfcc44ff491aad83a51135079315.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pc3MxMTgxMjQ4OTgz_size_16_color_FFFFFF_t_70 6]: /images/20220317/1a4278f7ff3c494184a4b3573ce89a10.png