[K8S] StatefulSet

柔情只为你懂 2022-09-09 04:54 129阅读 0赞

@Service DNS  
CoreDNS服务为每一个Service创建DNS记录用于域名解析  
ClusterIP记录格式：<service-name>.<namespace-name>.svc.cluster.local

[root@k8s-master ~]# kubectl get pod -n kube-system| grep dns
    coredns-7f89b7bc75-mgtnj                  1/1     Running   24         34d
    coredns-7f89b7bc75-wkrjq                  1/1     Running   24         34d
    [root@k8s-master ~]# kubectl exec -it coredns-7f89b7bc75-mgtnj -n kube-system -- bash
    OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "bash": executable file not found in $PATH: unknown
    command terminated with exit code 126
    [root@k8s-master ~]# kubectl exec -it coredns-7f89b7bc75-mgtnj -n kube-system -- sh
    OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "sh": executable file not found in $PATH: unknown
    command terminated with exit code 126
    [root@k8s-master ~]#

由于coredns pod进不去bash，部署一个busybox看看

[root@k8s-master ~]# kubectl run dns --image=busybox:1.28.4 -- sleep 3600
    pod/dns created
    [root@k8s-master ~]#

默认命名空间有一个kubernetes,用nslookup试试

[root@k8s-master ~]# kubectl get svc
    NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
    kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   34d
    [root@k8s-master ~]#
    [root@k8s-master ~]# kubectl get pod
    NAME                                      READY   STATUS    RESTARTS   AGE
    dns                                       1/1     Running   0          65s
    nfs-client-provisioner-5fd446cd9d-c62kl   1/1     Running   0          5h10m
    [root@k8s-master ~]#
    [root@k8s-master ~]# kubectl exec -it dns -- sh
    / # nslookup kubernetes
    Server:    10.96.0.10
    Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
    
    Name:      kubernetes
    Address 1: 10.96.0.1 kubernetes.default.svc.cluster.local
    / # exit
    [root@k8s-master ~]#

再看看命名空间kube-system,发现如果不加命名空间找不到,这个格式越完整搜索范围越小

[root@k8s-master ~]# kubectl get svc -n kube-system
    NAME             TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                  AGE
    kube-dns         ClusterIP   10.96.0.10      <none>        53/UDP,53/TCP,9153/TCP   34d
    metrics-server   ClusterIP   10.96.228.107   <none>        443/TCP                  34d
    [root@k8s-master ~]#
    [root@k8s-master ~]# kubectl exec -it dns -- sh
    / # nslookup kube-dns
    Server:    10.96.0.10
    Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
    
    nslookup: can't resolve 'kube-dns'
    / # nslookup metrics-server
    Server:    10.96.0.10
    Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
    
    nslookup: can't resolve 'metrics-server'
    / #

加上命名空间 或者 搜索ClusterIP就能找到

/ # nslookup kube-dns.kube-system
    Server:    10.96.0.10
    Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
    
    Name:      kube-dns.kube-system
    Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
    / #
    / # nslookup metrics-server.kube-system
    Server:    10.96.0.10
    Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
    
    Name:      metrics-server.kube-system
    Address 1: 10.96.228.107 metrics-server.kube-system.svc.cluster.local
    / #

@ StatefulSet控制器  
Deployment中Pod是平等的,StatefulSet中的Pod不等价且有序(例如mysql的读写分离)，所以  
\- StatefulSet不通过ClusterIP访问，使用无头服务(Headless Service)  
\- StatefulSet的存储卷使用卷申领模板(VolumeClaimTemplate)创建，和PVC类似，但是会为每个Pod创建一个PVC并分配编号  
可以参考官网的yaml ->  
https://kubernetes.io/zh/docs/tutorials/stateful-application/basic-stateful-set/

test-statefulset.yaml 内容如下,

apiVersion: v1
    kind: Service
    metadata:
      name: web-stat
      labels:
        app: test-statefulset
    spec:
      ports:
      - port: 80
        name: web-stat
      clusterIP: None
      selector:
        app: test-statefulset
    ---
    apiVersion: apps/v1
    kind: StatefulSet
    metadata:
      name: web
    spec:
      serviceName: "web-stat"
      replicas: 3
      selector:
        matchLabels:
          app: test-statefulset
      template:
        metadata:
          labels:
            app: test-statefulset
        spec:
          containers:
          - name: web-stat
            image: nginx
            ports:
            - containerPort: 80
              name: web-stat
            volumeMounts:
            - name: www
              mountPath: /usr/share/nginx/html
      volumeClaimTemplates:
      - metadata:
          name: www
        spec:
          storageClassName: "managed-nfs-storage"
          accessModes: [ "ReadWriteOnce" ]
          resources:
            requests:
              storage: 1Gi

说明：  
line 11 资源SVC,clusterIP: None,不分配clusterIP使用无头服务  
line 20 资源StatefulSet的serviceName: "web-stat"和资源SVC的name: web-stat(line 4)要对应  
line 43 volumeClaimTemplates中定义存储使用动态共给storageClassName: "managed-nfs-storage"

创建Service, StatefulSet, 可以观察到 **(1) Pod是有序起来的,Pod名称后有一个编号**

[root@k8s-master ~]# kubectl apply -f test-statefulset.yaml
    service/web-stat created
    statefulset.apps/web created
    [root@k8s-master ~]# kubectl get pod
    NAME                                      READY   STATUS              RESTARTS   AGE
    dns                                       1/1     Running             0          57m
    nfs-client-provisioner-5fd446cd9d-c62kl   1/1     Running             0          6h7m
    web-0                                     1/1     Running             0          31s
    web-1                                     0/1     ContainerCreating   0          12s
    [root@k8s-master ~]# kubectl get pod
    NAME                                      READY   STATUS    RESTARTS   AGE
    dns                                       1/1     Running   0          57m
    nfs-client-provisioner-5fd446cd9d-c62kl   1/1     Running   0          6h7m
    web-0                                     1/1     Running   0          40s
    web-1                                     1/1     Running   0          21s
    web-2                                     0/1     Pending   0          1s
    [root@k8s-master ~]# kubectl get pod
    NAME                                      READY   STATUS    RESTARTS   AGE
    dns                                       1/1     Running   0          57m
    nfs-client-provisioner-5fd446cd9d-c62kl   1/1     Running   0          6h7m
    web-0                                     1/1     Running   0          47s
    web-1                                     1/1     Running   0          28s
    web-2                                     1/1     Running   0          8s
    [root@k8s-master ~]#

**(2) 所创建的SVC的的ClusterIP为None**

[root@k8s-master ~]# kubectl get svc
    NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
    kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   34d
    web-stat     ClusterIP   None         <none>        80/TCP    109s
    [root@k8s-master ~]#

**(3) nslookup这个无头服务,解析的是三个Pod的地址; 而()Deployment部署解析的是ClusterIP**

StaetefulSet:
    
    [root@k8s-master ~]# kubectl exec -it dns -- sh
    / # nslookup web
    Server:    10.96.0.10
    Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
    
    nslookup: can't resolve 'web'
    / # nslookup web-stat
    Server:    10.96.0.10
    Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
    
    Name:      web-stat
    Address 1: 10.244.36.96 web-0.web-stat.default.svc.cluster.local
    Address 2: 10.244.169.154 web-2.web-stat.default.svc.cluster.local
    Address 3: 10.244.36.91 web-1.web-stat.default.svc.cluster.local
    / # 
    
    
    -----------
    Deployment:
    
    [root@k8s-master ~]# kubectl apply -f test-deploy-svc.yaml
    deployment.apps/de-web created
    service/de-web created
    [root@k8s-master ~]# kubectl get svc
    NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
    de-web       NodePort    10.111.52.13   <none>        80:30006/TCP   65s
    kubernetes   ClusterIP   10.96.0.1      <none>        443/TCP        34d
    web-stat     ClusterIP   None           <none>        80/TCP         16m
    [root@k8s-master ~]# kubectl get ep
    NAME                                          ENDPOINTS                                            AGE
    de-web                                        10.244.169.151:80,10.244.36.100:80,10.244.36.98:80   68s
    k8s-sigs.io-nfs-subdir-external-provisioner   <none>                                               6h22m
    kubernetes                                    192.168.231.121:6443                                 34d
    web-stat                                      10.244.169.154:80,10.244.36.91:80,10.244.36.96:80    16m
    [root@k8s-master ~]#
    [root@k8s-master ~]# kubectl exec -it dns -- sh
    / # nslookup de-web
    Server:    10.96.0.10
    Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
    
    Name:      de-web
    Address 1: 10.111.52.13 de-web.default.svc.cluster.local
    / # exit
    [root@k8s-master ~]#

Service ClusterIP记录格式：<service-name>.<namespace-name>.svc.cluster.local  
无头服务地址记录格式: <pod-name>.<service-name>.<namespace-name>.svc.cluster.local

**(4) 三个Pod对应三个PVC, 各自绑定一个PV, 存储互不干扰**

[root@k8s-master ~]# kubectl get pvc
    NAME        STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS          AGE
    www-web-0   Bound    pvc-d300aaa3-b98c-4d16-9bc5-76b4df012a67   1Gi        RWO            managed-nfs-storage   2m14s
    www-web-1   Bound    pvc-b8035a5d-8ce7-4b11-b253-d97c1f8dc8d9   1Gi        RWO            managed-nfs-storage   115s
    www-web-2   Bound    pvc-c129ccb2-69d2-468f-b98f-74ce490b9822   1Gi        RWO            managed-nfs-storage   95s
    [root@k8s-master ~]#
    [root@k8s-master ~]# kubectl get pv
    NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM               STORAGECLASS          REASON   AGE
    pvc-b8035a5d-8ce7-4b11-b253-d97c1f8dc8d9   1Gi        RWO            Delete           Bound    default/www-web-1   managed-nfs-storage            119s
    pvc-c129ccb2-69d2-468f-b98f-74ce490b9822   1Gi        RWO            Delete           Bound    default/www-web-2   managed-nfs-storage            99s
    pvc-d300aaa3-b98c-4d16-9bc5-76b4df012a67   1Gi        RWO            Delete           Bound    default/www-web-0   managed-nfs-storage            2m18s
    [root@k8s-master ~]#
    
    
    NFS:
    [root@k8s-node2 nfstest]# ls -l
    total 0
    drwxrwxrwx 2 root root 22 Aug 29 12:07 archived-default-test-sc-pvc-7b7c6f87-95e2-4d58-97af-b06723154615
    drwxrwxrwx 2 root root  6 Aug 29 17:07 default-www-web-0-pvc-d300aaa3-b98c-4d16-9bc5-76b4df012a67
    drwxrwxrwx 2 root root  6 Aug 29 17:07 default-www-web-1-pvc-b8035a5d-8ce7-4b11-b253-d97c1f8dc8d9
    drwxrwxrwx 2 root root  6 Aug 29 17:07 default-www-web-2-pvc-c129ccb2-69d2-468f-b98f-74ce490b9822
    [root@k8s-node2 nfstest]#