如何在Kubernetes集群动态使用 NAS 持久卷

傷城~ 2022-03-18 09:15 288阅读 0赞

### 1. 介绍： ###

本文介绍的动态生成NAS存储卷的方案：在一个已有文件系统上，自动生成一个目录，这个目录定义为目标存储卷；

镜像地址：registry.cn-hangzhou.aliyuncs.com/acs/alicloud-nas-controller:v1.11.5.4-433631d-aliyun

默认生成资源：  
生成的PV名字为：pvc-$\{pvc-uid\}  
生成目录的名字：namespace-pvcname-pvname

可以再pvc的annotations中如下声明，自定义名字：  
生成的pv、目录名字为下面定义的名字。

annotations:
        pv-name-created: replace-user-id

### 2. 部署NAS Controller ###

创建alicloud-nas-controller，实现动态provider nas pv；  
创建alicloud-nas storageclass，为nas pv provision 提供模板；

apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: alicloud-nas
    provisioner: alicloud/nas
    reclaimPolicy: Delete
    parameters:
      drivertype: flexvolume
      nfsversion: "4.0"
      options: ""
    
    ---
    kind: Deployment
    apiVersion: extensions/v1beta1
    metadata:
      name: alicloud-nas-controller
      namespace: kube-system
    spec:
      replicas: 1
      strategy:
        type: Recreate
      template:
        metadata:
          labels:
            app: alicloud-nas-controller
        spec:
          tolerations:
          - effect: NoSchedule
            operator: Exists
            key: node-role.kubernetes.io/master
          - effect: NoSchedule
            operator: Exists
            key: node.cloudprovider.kubernetes.io/uninitialized
          serviceAccount: admin
          containers:
            - name: alicloud-nas-controller
              image: registry.cn-hangzhou.aliyuncs.com/acs/alicloud-nas-controller:v1.11.5.4-433631d-aliyun
              imagePullPolicy: Always
              volumeMounts:
              - mountPath: /persistentvolumes
                name: nfs-client-root
              env:
                - name: NFS_SERVER
                  value: 154154b095-**.cn-beijing.nas.aliyuncs.com
                - name: NFS_PATH
                  value: /
          volumes:
          - name: nfs-client-root
            flexVolume:
              driver: alicloud/nas
              options:
                path: /
                server: 154154b095-**.cn-beijing.nas.aliyuncs.com
                vers: "4.0"

StorageClass使用说明：

drivertype: 用来表示生成pv存储类型，可选nfs, flexvolume.
        nfs: 默认选项，表示使用k8s原生NFS驱动挂载；
        flexvolume: 表示使用阿里云提供的Flexvolume NAS驱动挂载；
    
    nfsversion: 挂载nfs使用的版本，支持3，4.0.默认为4.0；
        drivertype为flexvolume的时候在这里配置；
        为nfs的时候通过mountOptions 配置；
    
    options：为挂载nfs的可选项配置；
        drivertype为flexvolume的时候在这里配置；
        为nfs的时候通过mountOptions 配置；

StorageClass举例：

## 使用kubernetes提供的NFS驱动，并配置mountOptions，reclaimPolicy为Delete；
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: alicloud-nas-nfs
    mountOptions:
    - vers=4.0
    - noresvport
    provisioner: alicloud/nas
    reclaimPolicy: Delete
    
    ## 使用阿里云提供的Flexvolume NAS驱动，配置nfs版本、options；
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: alicloud-nas-flex
    provisioner: alicloud/nas
    reclaimPolicy: Delete
    parameters:
      drivertype: flexvolume
      nfsversion: "3"
      options: "noresvport"

### 3. 创建应用-Deployment： ###

kind: PersistentVolumeClaim
    apiVersion: v1
    metadata:
      name: replace-user-id
      annotations:
        pv-name-created: replace-user-id
    spec:
      storageClassName: alicloud-nas
      accessModes:
        - ReadWriteMany
      resources:
        requests:
          storage: 5Gi
    ---
    apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
      name: "deploy-nas"
    spec:
      replicas: 1
      strategy:
        type: Recreate
      template:
        metadata:
          labels:
            app: deploy-nas
        spec:
          containers:
            - name: "nginx"
              image: "nginx"
              volumeMounts:
                  - name: pvc-nas
                    mountPath: "/data"
          volumes:
          - name: pvc-nas
            persistentVolumeClaim:
                claimName: replace-user-id
    
    执行：
    # userID="hello-123"
    # cat deploy.yaml | sed  "s/replace-user-id/\"$userID\"/g" | kubectl create -f -
    
    # kubectl get pod | grep deploy-nas
    deploy-nas-85696b6bfc-t5dmh         1/1       Running          0          28m
    
    # kubectl get pvc | grep hell
    hello-123       Bound     hello-123                                  5Gi        RWX            alicloud-nas-flex   28m
    
    # kubectl get pv | grep hell
    hello-123                                  5Gi        RWX            Delete           Bound      default/hello-123       alicloud-nas-flex                       28m
    
    # Nas目录下查看生成目录：
    # ls -l | grep hello
    drwxrwxrwx  2 root root 4096 2月  19 09:58 hello-123

### 4. 创建应用-StatefulSet： ###

使用volumeTemplateClaim不支持使用pv-name-created配置pv名字；

apiVersion: v1
    kind: Service
    metadata:
      name: nginx
      labels:
        app: nginx
    spec:
      ports:
      - port: 80
        name: web
      clusterIP: None
      selector:
        app: nginx
    ---
    apiVersion: apps/v1beta1
    kind: StatefulSet
    metadata:
      name: web
    spec:
      replicas: 2
      serviceName: "nginx"
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          - name: nginx
            image: nginx:alpine
            volumeMounts:
            - mountPath: "/data"
              name: pvc-sts
      volumeClaimTemplates:
      - metadata:
          name: pvc-sts
        spec:
          accessModes:
            - ReadWriteOnce
          storageClassName: alicloud-nas-flex
          resources:
            requests:
              storage: 2Gi
              
    创建后查看：
    # kubectl get pod | grep web
    web-0                               1/1       Running          0          7s
    web-1                               1/1       Running          0          4s
    
    # kubectl get pvc | grep web
    pvc-sts-web-0   Bound     pvc-65ab251a-33ec-11e9-a151-00163e066784   2Gi        RWO            alicloud-nas-flex   13m
    pvc-sts-web-1   Bound     pvc-8437c50e-33ed-11e9-a151-00163e066784   2Gi        RWO            alicloud-nas-flex   5m
    
    # kubectl get pv | grep web
    pvc-65ab251a-33ec-11e9-a151-00163e066784   2Gi        RWO            Delete           Bound      default/pvc-sts-web-0   alicloud-nas-flex                       13m
    pvc-8437c50e-33ed-11e9-a151-00163e066784   2Gi        RWO            Delete           Bound      default/pvc-sts-web-1   alicloud-nas-flex                       5m
    
    # Nas目录下查看生成目录：
    # ls -l | grep sts
    drwxrwxrwx  2 root root 4096 2月  19 10:16 default-pvc-sts-web-0-pvc-65ab251a-33ec-11e9-a151-00163e066784
    drwxrwxrwx  2 root root 4096 2月  19 10:24 default-pvc-sts-web-1-pvc-8437c50e-33ed-11e9-a151-00163e066784

### 5. 创建应用-Pod： ###

kind: PersistentVolumeClaim
    apiVersion: v1
    metadata:
      name: replace-user-id
      annotations:
        pv-name-created: replace-user-id
    spec:
      storageClassName: alicloud-nas-flex
      accessModes:
        - ReadWriteMany
      resources:
        requests:
          storage: 5Gi
    ---
    apiVersion: v1
    kind: Pod
    metadata:
      name: "nas-pod"
    spec:
      containers:
        - name: "nginx"
          image: "nginx"
          volumeMounts:
              - name: pvc-nas
                mountPath: "/data"
      volumes:
      - name: pvc-nas
        persistentVolumeClaim:
            claimName: replace-user-id
           
    # userID="pod-123"
    # cat pod.yaml | sed  "s/replace-user-id/\"$userID\"/g" | kubectl create -f -
    
    # kubectl get pod | grep pod
    nas-pod                             1/1       Running          0          32s
    
    # kubectl get pvc | grep pod
    pod-123         Bound     pod-123                                    5Gi        RWX            alicloud-nas-flex   44s
    
    # kubectl get pv | grep pod
    pod-123                                    5Gi        RWX            Delete           Bound      default/pod-123         alicloud-nas-flex                       48s
    
    # ls -l | grep pod
    drwxrwxrwx  2 root root 4096 2月  19 10:54 pod-123

--------------------

本文作者：kanjunbao

[阅读原文][Link 1]

本文为云栖社区原创内容，未经允许不得转载。

[Link 1]: https://yq.aliyun.com/articles/690679?utm_content=g_1000041862