Kubernetes 核心实战之三（精华篇 3/3）

亦凉 2024-02-05 22:58 8阅读 0赞

#### 文章目录 ####

*   *   *  6、Ingress ★
         *   *  6.1 安装 Ingress
             *  6.2 访问
             *  6.3 安装不成功的bug解决
             *  6.4 测试使用
             *   *  6.4.1 搭建测试环境
                 *  6.4.2 配置 Ingress的规则
                 *  6.4.3 测试I
                 *  6.4.4 测试II
                 *  6.4.5 路径重写
                 *  6.4.6 限流
         *  7. Kubernetes 存储抽象
         *   *  7.1 NFS 搭建
             *  7.2 原生方式 数据挂载
             *  7.3 PV 和 PVC ★
             *   *  7.3.1 创建 PV 池
                 *  7.3.2 创建、绑定 PCV
                 *  7.3.3 创建 Pod 绑定 PVC
             *  7.4 ConfigMap ★
             *   *  7.4.1 redis示例
             *  7.5 Secret
             *   *  7.5.1 拉取失败
                 *  7.5.2 创建 Secret

#### 6、Ingress ★ ####

Ingress：Service 的统一网关入口，底层就是 nginx。（服务）

官网地址：[https://kubernetes.github.io/ingress-nginx/][https_kubernetes.github.io_ingress-nginx] （都是从这里看的）

所有的请求都先通过 Ingress，由 Ingress 来 打理这些请求。类似微服务中的 网关。

![在这里插入图片描述][c6c9c7b12d0b424ab5177fbb5f5ba904.png]

##### 6.1 安装 Ingress #####

wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.47.0/deploy/static/provider/baremetal/deploy.yaml
    
    ## 这里我喜欢把depoly.yaml名字修改为ingress.yaml
    
    # 修改镜像
    vi ingress.yaml
    # 将 image 的值改为如下值
    registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/ingress-nginx-controller:v0.46.0
    
    # 安装资源
    kubectl apply -f ingress.yaml
    
    # 检查安装的结果
    kubectl get pod,svc -n ingress-nginx
    
    # 最后别忘记把 svc 暴露的端口 在安全组放行

ingress.yaml整个文件内容：

apiVersion: v1
    kind: Namespace
    metadata:
      name: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
    
    ---
    # Source: ingress-nginx/templates/controller-serviceaccount.yaml
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: controller
      name: ingress-nginx
      namespace: ingress-nginx
    automountServiceAccountToken: true
    ---
    # Source: ingress-nginx/templates/controller-configmap.yaml
    apiVersion: v1
    kind: ConfigMap
    metadata:
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: controller
      name: ingress-nginx-controller
      namespace: ingress-nginx
    data:
    ---
    # Source: ingress-nginx/templates/clusterrole.yaml
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
      name: ingress-nginx
    rules:
      - apiGroups:
          - ''
        resources:
          - configmaps
          - endpoints
          - nodes
          - pods
          - secrets
        verbs:
          - list
          - watch
      - apiGroups:
          - ''
        resources:
          - nodes
        verbs:
          - get
      - apiGroups:
          - ''
        resources:
          - services
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - extensions
          - networking.k8s.io   # k8s 1.14+
        resources:
          - ingresses
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - ''
        resources:
          - events
        verbs:
          - create
          - patch
      - apiGroups:
          - extensions
          - networking.k8s.io   # k8s 1.14+
        resources:
          - ingresses/status
        verbs:
          - update
      - apiGroups:
          - networking.k8s.io   # k8s 1.14+
        resources:
          - ingressclasses
        verbs:
          - get
          - list
          - watch
    ---
    # Source: ingress-nginx/templates/clusterrolebinding.yaml
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
      name: ingress-nginx
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: ingress-nginx
    subjects:
      - kind: ServiceAccount
        name: ingress-nginx
        namespace: ingress-nginx
    ---
    # Source: ingress-nginx/templates/controller-role.yaml
    apiVersion: rbac.authorization.k8s.io/v1
    kind: Role
    metadata:
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: controller
      name: ingress-nginx
      namespace: ingress-nginx
    rules:
      - apiGroups:
          - ''
        resources:
          - namespaces
        verbs:
          - get
      - apiGroups:
          - ''
        resources:
          - configmaps
          - pods
          - secrets
          - endpoints
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - ''
        resources:
          - services
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - extensions
          - networking.k8s.io   # k8s 1.14+
        resources:
          - ingresses
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - extensions
          - networking.k8s.io   # k8s 1.14+
        resources:
          - ingresses/status
        verbs:
          - update
      - apiGroups:
          - networking.k8s.io   # k8s 1.14+
        resources:
          - ingressclasses
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - ''
        resources:
          - configmaps
        resourceNames:
          - ingress-controller-leader-nginx
        verbs:
          - get
          - update
      - apiGroups:
          - ''
        resources:
          - configmaps
        verbs:
          - create
      - apiGroups:
          - ''
        resources:
          - events
        verbs:
          - create
          - patch
    ---
    # Source: ingress-nginx/templates/controller-rolebinding.yaml
    apiVersion: rbac.authorization.k8s.io/v1
    kind: RoleBinding
    metadata:
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: controller
      name: ingress-nginx
      namespace: ingress-nginx
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: Role
      name: ingress-nginx
    subjects:
      - kind: ServiceAccount
        name: ingress-nginx
        namespace: ingress-nginx
    ---
    # Source: ingress-nginx/templates/controller-service-webhook.yaml
    apiVersion: v1
    kind: Service
    metadata:
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: controller
      name: ingress-nginx-controller-admission
      namespace: ingress-nginx
    spec:
      type: ClusterIP
      ports:
        - name: https-webhook
          port: 443
          targetPort: webhook
      selector:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/component: controller
    ---
    # Source: ingress-nginx/templates/controller-service.yaml
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: controller
      name: ingress-nginx-controller
      namespace: ingress-nginx
    spec:
      type: NodePort
      ports:
        - name: http
          port: 80
          protocol: TCP
          targetPort: http
        - name: https
          port: 443
          protocol: TCP
          targetPort: https
      selector:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/component: controller
    ---
    # Source: ingress-nginx/templates/controller-deployment.yaml
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: controller
      name: ingress-nginx-controller
      namespace: ingress-nginx
    spec:
      selector:
        matchLabels:
          app.kubernetes.io/name: ingress-nginx
          app.kubernetes.io/instance: ingress-nginx
          app.kubernetes.io/component: controller
      revisionHistoryLimit: 10
      minReadySeconds: 0
      template:
        metadata:
          labels:
            app.kubernetes.io/name: ingress-nginx
            app.kubernetes.io/instance: ingress-nginx
            app.kubernetes.io/component: controller
        spec:
          dnsPolicy: ClusterFirst
          containers:
            - name: controller
              image: registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/ingress-nginx-controller:v0.46.0
              imagePullPolicy: IfNotPresent
              lifecycle:
                preStop:
                  exec:
                    command:
                      - /wait-shutdown
              args:
                - /nginx-ingress-controller
                - --election-id=ingress-controller-leader
                - --ingress-class=nginx
                - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
                - --validating-webhook=:8443
                - --validating-webhook-certificate=/usr/local/certificates/cert
                - --validating-webhook-key=/usr/local/certificates/key
              securityContext:
                capabilities:
                  drop:
                    - ALL
                  add:
                    - NET_BIND_SERVICE
                runAsUser: 101
                allowPrivilegeEscalation: true
              env:
                - name: POD_NAME
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.name
                - name: POD_NAMESPACE
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.namespace
                - name: LD_PRELOAD
                  value: /usr/local/lib/libmimalloc.so
              livenessProbe:
                failureThreshold: 5
                httpGet:
                  path: /healthz
                  port: 10254
                  scheme: HTTP
                initialDelaySeconds: 10
                periodSeconds: 10
                successThreshold: 1
                timeoutSeconds: 1
              readinessProbe:
                failureThreshold: 3
                httpGet:
                  path: /healthz
                  port: 10254
                  scheme: HTTP
                initialDelaySeconds: 10
                periodSeconds: 10
                successThreshold: 1
                timeoutSeconds: 1
              ports:
                - name: http
                  containerPort: 80
                  protocol: TCP
                - name: https
                  containerPort: 443
                  protocol: TCP
                - name: webhook
                  containerPort: 8443
                  protocol: TCP
              volumeMounts:
                - name: webhook-cert
                  mountPath: /usr/local/certificates/
                  readOnly: true
              resources:
                requests:
                  cpu: 100m
                  memory: 90Mi
          nodeSelector:
            kubernetes.io/os: linux
          serviceAccountName: ingress-nginx
          terminationGracePeriodSeconds: 300
          volumes:
            - name: webhook-cert
              secret:
                secretName: ingress-nginx-admission
    ---
    # Source: ingress-nginx/templates/admission-webhooks/validating-webhook.yaml
    # before changing this value, check the required kubernetes version
    # https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#prerequisites
    apiVersion: admissionregistration.k8s.io/v1
    kind: ValidatingWebhookConfiguration
    metadata:
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: admission-webhook
      name: ingress-nginx-admission
    webhooks:
      - name: validate.nginx.ingress.kubernetes.io
        matchPolicy: Equivalent
        rules:
          - apiGroups:
              - networking.k8s.io
            apiVersions:
              - v1beta1
            operations:
              - CREATE
              - UPDATE
            resources:
              - ingresses
        failurePolicy: Fail
        sideEffects: None
        admissionReviewVersions:
          - v1
          - v1beta1
        clientConfig:
          service:
            namespace: ingress-nginx
            name: ingress-nginx-controller-admission
            path: /networking/v1beta1/ingresses
    ---
    # Source: ingress-nginx/templates/admission-webhooks/job-patch/serviceaccount.yaml
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: ingress-nginx-admission
      annotations:
        helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
        helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: admission-webhook
      namespace: ingress-nginx
    ---
    # Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrole.yaml
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
      name: ingress-nginx-admission
      annotations:
        helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
        helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: admission-webhook
    rules:
      - apiGroups:
          - admissionregistration.k8s.io
        resources:
          - validatingwebhookconfigurations
        verbs:
          - get
          - update
    ---
    # Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrolebinding.yaml
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      name: ingress-nginx-admission
      annotations:
        helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
        helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: admission-webhook
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: ingress-nginx-admission
    subjects:
      - kind: ServiceAccount
        name: ingress-nginx-admission
        namespace: ingress-nginx
    ---
    # Source: ingress-nginx/templates/admission-webhooks/job-patch/role.yaml
    apiVersion: rbac.authorization.k8s.io/v1
    kind: Role
    metadata:
      name: ingress-nginx-admission
      annotations:
        helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
        helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: admission-webhook
      namespace: ingress-nginx
    rules:
      - apiGroups:
          - ''
        resources:
          - secrets
        verbs:
          - get
          - create
    ---
    # Source: ingress-nginx/templates/admission-webhooks/job-patch/rolebinding.yaml
    apiVersion: rbac.authorization.k8s.io/v1
    kind: RoleBinding
    metadata:
      name: ingress-nginx-admission
      annotations:
        helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
        helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: admission-webhook
      namespace: ingress-nginx
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: Role
      name: ingress-nginx-admission
    subjects:
      - kind: ServiceAccount
        name: ingress-nginx-admission
        namespace: ingress-nginx
    ---
    # Source: ingress-nginx/templates/admission-webhooks/job-patch/job-createSecret.yaml
    apiVersion: batch/v1
    kind: Job
    metadata:
      name: ingress-nginx-admission-create
      annotations:
        helm.sh/hook: pre-install,pre-upgrade
        helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: admission-webhook
      namespace: ingress-nginx
    spec:
      template:
        metadata:
          name: ingress-nginx-admission-create
          labels:
            helm.sh/chart: ingress-nginx-3.33.0
            app.kubernetes.io/name: ingress-nginx
            app.kubernetes.io/instance: ingress-nginx
            app.kubernetes.io/version: 0.47.0
            app.kubernetes.io/managed-by: Helm
            app.kubernetes.io/component: admission-webhook
        spec:
          containers:
            - name: create
              image: docker.io/jettech/kube-webhook-certgen:v1.5.1
              imagePullPolicy: IfNotPresent
              args:
                - create
                - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
                - --namespace=$(POD_NAMESPACE)
                - --secret-name=ingress-nginx-admission
              env:
                - name: POD_NAMESPACE
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.namespace
          restartPolicy: OnFailure
          serviceAccountName: ingress-nginx-admission
          securityContext:
            runAsNonRoot: true
            runAsUser: 2000
    ---
    # Source: ingress-nginx/templates/admission-webhooks/job-patch/job-patchWebhook.yaml
    apiVersion: batch/v1
    kind: Job
    metadata:
      name: ingress-nginx-admission-patch
      annotations:
        helm.sh/hook: post-install,post-upgrade
        helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: admission-webhook
      namespace: ingress-nginx
    spec:
      template:
        metadata:
          name: ingress-nginx-admission-patch
          labels:
            helm.sh/chart: ingress-nginx-3.33.0
            app.kubernetes.io/name: ingress-nginx
            app.kubernetes.io/instance: ingress-nginx
            app.kubernetes.io/version: 0.47.0
            app.kubernetes.io/managed-by: Helm
            app.kubernetes.io/component: admission-webhook
        spec:
          containers:
            - name: patch
              image: docker.io/jettech/kube-webhook-certgen:v1.5.1
              imagePullPolicy: IfNotPresent
              args:
                - patch
                - --webhook-name=ingress-nginx-admission
                - --namespace=$(POD_NAMESPACE)
                - --patch-mutating=false
                - --secret-name=ingress-nginx-admission
                - --patch-failure-policy=Fail
              env:
                - name: POD_NAMESPACE
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.namespace
          restartPolicy: OnFailure
          serviceAccountName: ingress-nginx-admission
          securityContext:
            runAsNonRoot: true
            runAsUser: 2000

![在这里插入图片描述][a239b51524a044eab3a44f783b3dc738.png]  
耐心等待pod创建完成，所有状态变成running。

开放青云服务器端口 30000-32767  
![在这里插入图片描述][90998cb8b1e5495f8446a5792d5b45c7.png]  
查看 映射后的 IP

kubectl get pod,svc -n ingress-nginx

![在这里插入图片描述][daa733d8980e45e8aa1c3d1bd2d02a9d.png]

##### 6.2 访问 #####

每台服务器 都开放 映射后的 端口：  
31737、32641

https://xxxxxxxx:32641  
http://xxxxxxxxx:31737

![在这里插入图片描述][06125fc4d9034ca8b15c97fec1f7c11a.png]

##### 6.3 安装不成功的bug解决 #####

查看pod，一直在创建中。  
![在这里插入图片描述][55deb9e1808b4ae69518759f439dae4f.png]  
使用describe，查看详细问题日志：

kubectl describe pod ingress-nginx-controller-65bf56f7fc-nwgff -n ingress-nginx

![在这里插入图片描述][3d1f079fd0f34f258548c23e35b8441f.png]  
解决办法：  
![在这里插入图片描述][e989b076c0704cb388dbf660fdf01443.png]  
![在这里插入图片描述][3c6626272d0e4137a21752e8c06b9fd4.png]  
![在这里插入图片描述][62a68d78314b48a6a7bcc24b6b4f8f94.png]  
成功启动：  
![在这里插入图片描述][d17b8e0a8dd542e79e5bd88180519957.png]  
重新查看端口并访问  
![在这里插入图片描述][1ddc6ecf6df645e3915db7ef12f17aa7.png]  
![在这里插入图片描述][1edb2ae49ff740e5949b0e120dc22bff.png]  
![在这里插入图片描述][5868c7642c1c46cc8173fac994669347.png]

##### 6.4 测试使用 #####

官网地址：[https://kubernetes.github.io/ingress-nginx/][https_kubernetes.github.io_ingress-nginx]  
ingress底层其实就是nginx。

###### 6.4.1 搭建测试环境 ######

创建两个 Service和deployment

vi test.yaml
    
    # 复制下面
    
    kubectl apply -f test.yaml

test.yaml整个文件内容：

apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: hello-server
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: hello-server
      template:
        metadata:
          labels:
            app: hello-server
        spec:
          containers:
          - name: hello-server
            image: registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/hello-server
            ports:
            - containerPort: 9000
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: nginx-demo
      name: nginx-demo
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: nginx-demo
      template:
        metadata:
          labels:
            app: nginx-demo
        spec:
          containers:
          - image: nginx
            name: nginx
    ---
    apiVersion: v1
    kind: Service
    metadata:
      labels:
        app: nginx-demo
      name: nginx-demo
    spec:
      selector:
        app: nginx-demo
      ports:
      - port: 8000
        protocol: TCP
        targetPort: 80
    ---
    apiVersion: v1
    kind: Service
    metadata:
      labels:
        app: hello-server
      name: hello-server
    spec:
      selector:
        app: hello-server
      ports:
      - port: 8000
        protocol: TCP
        targetPort: 9000

![在这里插入图片描述][543120835d1f40b7ae38e0883f83d9cd.png]  
![在这里插入图片描述][1b34d2824980464fa8703caded69b6b3.png]

###### 6.4.2 配置 Ingress的规则 ######

vi ingress-rule.yaml
    
    # 复制下面配置
    
    kubectl apply -f ingress-rule.yaml
    
    # 查看 集群中的 Ingress
    kubectl get ingress

ingress-rule.yaml完整内容：

apiVersion: networking.k8s.io/v1
    kind: Ingress  
    metadata:
      name: ingress-host-bar
    spec:
      ingressClassName: nginx
      rules:
      - host: "hello.tigerhhzz.com"
        http:
          paths:
          - pathType: Prefix
            path: "/"
            backend:
              service:
                name: hello-server
                port:
                  number: 8000 # hello-server （service） 的端口是 8000
      - host: "demo.tigerhhzz.com"
        http:
          paths:
          - pathType: Prefix
            path: "/"  # 把请求会转给下面的服务，下面的服务一定要能处理这个路径，不能处理就是404
            backend:
              service:
                name: nginx-demo  #java，比如使用路径重写，去掉前缀nginx
                port:
                  number: 8000
    
    
    apiVersion: networking.k8s.io/v1
    kind: Ingress  
    metadata:
      name: ingress-host-bar
    spec:
      ingressClassName: nginx
      rules:
        - host: "hello.tigerhhzz.com"
          http:
            paths:
              - pathType: Prefix
                path: "/"
                backend:
                  service:
                    name: hello-server
                    port:
                      number: 8000 # hello-server 的端口是 8000
        - host: "demo.tigerhhzz.com"
          http:
            paths:
              - pathType: Prefix
                path: "/"
                backend:
                  service:
                    name: nginx-demo
                    port:
                      number: 8000

![在这里插入图片描述][3b4b76aa07264e029a511f2624144268.png]

解决上面错误的方法：

kubectl get ValidatingWebhookConfiguration
    # 把该死的 admission 删掉
    kubectl delete -A ValidatingWebhookConfiguration ingress-nginx-admission

![在这里插入图片描述][631cad721260446f87e5f0a1d95560e1.png]

###### 6.4.3 测试I ######

在 自己电脑（不是虚拟机） hosts 中增加映射：  
master的公网IP hello.tigerhhzz.com  
master的公网IP demo.tigerhhzz.com

![在这里插入图片描述][ee2d73b33e8d424789e7cfa615ca0563.png]  
![在这里插入图片描述][e736170a2c4d423e90a8b2b5c1035330.png]  
![在这里插入图片描述][151afef67e6643329f27d28af04bf516.png]

###### 6.4.4 测试II ######

# kubectl get ingress
    kubectl get ing
    
    kubectl edit ing ingress的NAME -n
    
    # 改变匹配的 path
      - host: "demo.tigerhhzz.com"
        http:
          paths:
          - pathType: Prefix
            path: "/nginx" # 匹配请求 /nginx 的，并且查找 nginx 文件.
            backend:
              service:
                name: nginx-demo
                port:
                  number: 8000

![在这里插入图片描述][a190c8623f7f4116b869b27bc875234d.png]  
随便写 /xxx 不匹配 nginx的，都返回 Ingress的 404的nginx  
![在这里插入图片描述][e78e7e49256049d18fe461668e7903a2.png]  
下面这个是 通过了 Ingress，Service 里的 Pod 没匹配到，才返回的 404（下面打印的 nginx 版本不一样的）  
![在这里插入图片描述][64640333f8aa4de7bd2ff3c0d46eb083.png]

页面 进入 Pod 的那个nginx

cd /usr/share/nginx/html
    ls
    echo "hello tigerhhzz" > nginx

![在这里插入图片描述][adcfbb51429b45c1ae4b5494ef432017.png]  
![在这里插入图片描述][5c2801184c99416b9854ed3918659f81.png]  
![在这里插入图片描述][d7b526fbff4243a8b06a550c7948ded3.png]  
![在这里插入图片描述][859b510207774be28caf80c2bc608e84.png]

###### 6.4.5 路径重写 ######

修改ingress配置规则，下面这个这样子 就和 SpringCloud Gateway 网管转发一样的效果了。  
ingress-rule.yaml的完整内容如下：

apiVersion: networking.k8s.io/v1
    kind: Ingress  
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/rewrite-target: /$2
      name: ingress-host-bar
    spec:
      ingressClassName: nginx
      rules:
      - host: "hello.tigerhhzz.com"
        http:
          paths:
          - pathType: Prefix
            path: "/"
            backend:
              service:
                name: hello-server
                port:
                  number: 8000
      - host: "demo.tigerhhzz.com"
        http:
          paths:
          - pathType: Prefix
            path: "/nginx(/|$)(.*)" 
            backend:
              service:
                name: nginx-demo  
                port:
                  number: 8000

在可视化界面 创建  
![在这里插入图片描述][024ae8a6727949939fd7276d91a75736.png]  
![在这里插入图片描述][42a56bc45fc6469296504f6158f06bbf.png]  
![在这里插入图片描述][8ffd9a0a9ef141958005745fc5cb0e51.png]

###### 6.4.6 限流 ######

官网文档：[https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration][https_kubernetes.github.io_ingress-nginx_user-guide_nginx-configuration]

![在这里插入图片描述][f16530ae471140268efb1e7878711ac0.png]

apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: ingress-limit-rate
      annotations:
        # 限流
        nginx.ingress.kubernetes.io/limit-rps: "1"
    spec:
      ingressClassName: nginx
      rules:
      - host: "haha.tigerhhzz.com"
        http:
          paths:
          - pathType: Exact
            path: "/"
            backend:
              service:
                name: nginx-demo
                port:
                  number: 8000

vim ingress-rule-2.yaml
    
    # 复制上面配置
    
    kubectl apply -f ingress-rule-2.yaml
    
    kubect get ing

在可视化界面 创建ingress  
![在这里插入图片描述][522a07b833854c3fb8dc9e40c267500e.png]  
![在这里插入图片描述][757a4f0566244948b083c461cf87fcb7.png]  
在 自己电脑（不是虚拟机） hosts 中增加映射：  
公网IP haha.tigerhhzz.com

访问测试：  
![在这里插入图片描述][fa900ebb41194383bb908f915ae4300e.png]  
http://haha.tigerhhzz.com:30808/

![在这里插入图片描述][917ad66897e542bba7ea73873bb61ba4.png]  
刷新过快 返回 503，官网文档也写了。  
![在这里插入图片描述][32b9d080ef08458d8f45882a33e4fcae.png]  
![在这里插入图片描述][21d970037120440fa2ab3ae2429e0e46.png]  
速率限制  
这些注释定义了连接和传输速率的限制。这些可用于缓解DDoS 攻击

*  nginx.ingress.kubernetes.io/limit-connections:允许来自单个IP 地址的并发连接数超过此限制时返回 503 错误
 *  nginx.ingress.kubernetes.io/limit-rps :每秒从给定IP 接受的请求数。突发限制设置为此限制乘以突发倍数，默认倍数为 5。当客户端超过此限制时，返回limit-reg-status-codedefault: 503
 *  nginx.ingress,kubernetes.io/limit-rpm:每分钟从给定IP 接受的请求数。突发限制设署为此限制乘以突发倍数，默认倍数为 5。当客户端超过此限制时，返回limit-reg-status-codedefault: 503
 *  nginx.ingress.kubernetes.io/limit-burst-multiplier:突发大小限制率的乘数。默认突发乘数为5，此注释覆盖默认乘数。当客户端超过此限制时，返回limit-req-status-codedefault: 503
 *  nginx.ingress.kubernetes.io/limit-rate-after :初始千字节数，之后对给定连接的响应的进一步传输将受到速率限制。此功能必须在启用代理缓冲的情况下使用
 *  nginx.ingress.kubernetes.io/limit-rate :每秒允许发送到给定连接的千字节数。零值禁用速率限制。此功能必须在启用代理缓冲的情况下使用。
 *  nginx.ingress.kubernetes.io/limit-whitelist :要从速率限制中排除的客户端IP源范围。该值是一个逗号分隔的 CIDR 列表

> 如果您在单个Ingress 规则中指定多个注释，则会按顺序应用限制  
> limit-connections，limit-rpm，limit-rps。
> 
> 要为所有Ingress 规则全局配置设置，可以在NGINX ConfigMap limit-rate-after  
> 中设置和值.Ingress注解中设置的值将覆盖全局设置。limit-rate
> 
> 客户端IP 地址将根据PROXY 协议的使用或启用use-forwarded-headers X-Forwarded-For  
> 时的标头值设置。

![在这里插入图片描述][517f221647994c8e828891133837c452.png]

#### 7. Kubernetes 存储抽象 ####

类似于 Docker 中的 挂载。但要考虑 自愈、故障转移 时的情况  
![在这里插入图片描述][0b2d84df440d44feabe3e30b2e9e1da9.png]

##### 7.1 NFS 搭建 #####

网络文件系统  
![在这里插入图片描述][047f9b7a618b4e9aa50dfb7321a2aada.png]  
1、所有节点  
安装nfs-utils

# 所有机器执行
    yum install -y nfs-utils

2、主节点

# 只在 mster 机器执行：nfs主节点，rw 读写
    echo "/nfs/data/ *(insecure,rw,sync,no_root_squash)" > /etc/exports
    
    mkdir -p /nfs/data
    systemctl enable rpcbind --now
    systemctl enable nfs-server --now
    
    # 配置生效
    exportfs -r

![在这里插入图片描述][b9e65882ad16415083143e401934ed30.png]  
3、从节点

# 检查，下面的 IP 是master IP
    showmount -e xxx.xxx.xxx.xxx
    
    # 在 2 个从服务器 执行，执行以下命令挂载 nfs 服务器上的共享目录到本机路径 /root/nfsmount
    mkdir -p /nfs/data
    
    # 在 2 个从服务器执行，将远程 和本地的 文件夹 挂载
    mount -t nfs 139.198.36.162:/nfs/data /nfs/data
    
    # 在 master 服务器，写入一个测试文件
    echo "hello nfs server" > /nfs/data/test.txt
    
    # 在 2 个从服务器查看
    cd /nfs/data
    ls
    
    # 在 从服务器 修改，然后去 其他 服务器 查看，也能 同步

##### 7.2 原生方式 数据挂载 #####

在 /nfs/data/nginx-pv 挂载，然后 修改， 里面 两个 Pod 也会 同步修改。  
问题：删掉之后，文件还在，内容也在，是没法管理大小的。

![在这里插入图片描述][f495f79f03d846d5b66cc846fe4e1e56.png]

apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: nginx-pv-demo
      name: nginx-pv-demo
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: nginx-pv-demo
      template:
        metadata:
          labels:
            app: nginx-pv-demo
        spec:
          containers:
          - image: nginx
            name: nginx
            volumeMounts:
            - name: html
              mountPath: /usr/share/nginx/html # 挂载目录
          volumes:
            # 和 volumeMounts.name 一样
            - name: html
              nfs:
                # master IP
                server: 192.168.27.251  #master节点ip
                path: /nfs/data/nginx-pv # 要提前创建好文件夹，否则挂载失败

cd /nfs/data
    mkdir -p nginx-pv
    ls
    
    vi deploy.yaml
    
    # 复制上面配置
    
    kubectl apply -f deploy.yaml
    
    kubectl get pod -owide

cd /nfs/data/
    ls
    cd nginx-pv/
    echo "cgxin" > index.html
    
    # 进入 pod 里面查看

问题：占用空间，删掉之后，文件还在，内容也在，是没法管理大小的。  
![在这里插入图片描述][05e5e564c75f406bb82bf00266f0ec39.png]

##### 7.3 PV 和 PVC ★ #####

PV：持久卷（Persistent Volume），将应用需要持久化的数据保存到指定位置  
PVC：持久卷申明（Persistent Volume Claim），申明需要使用的持久卷规格

挂载目录。ConfigMap 挂载配置文件。  
这里是 是 静态的， 就是自己创建好了 容量，然后 PVC 去挑。 还有 动态供应的，不用手动去创建 PV池子。

![在这里插入图片描述][a60b78ca6b7c4890981acb8bc68e01f3.png]

###### 7.3.1 创建 PV 池 ######

静态供应

# 在 nfs主节点（master服务器） 执行
    mkdir -p /nfs/data/01
    mkdir -p /nfs/data/02
    mkdir -p /nfs/data/03

使用pv.yaml 创建 3个 PV

apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: pv01-10m
    spec:
      # 限制容量
      capacity:
        storage: 10M
      # 读写模式：可读可写
      accessModes:
        - ReadWriteMany
      storageClassName: nfs
      nfs:
        # 挂载 上面创建过的文件夹
        path: /nfs/data/01
        # nfs 主节点服务器的 IP
        server: 139.198.36.162
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      # 这个name 要小写，如 Gi 大写就不行
      name: pv02-1gi
    spec:
      capacity:
        storage: 1gi
      accessModes:
        - ReadWriteMany
      storageClassName: nfs
      nfs:
        path: /nfs/data/02
        # nfs 主节点服务器的 IP
        server: 139.198.36.162
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: pv03-3gi
    spec:
      capacity:
        storage: 3gi
      accessModes:
        - ReadWriteMany
      storageClassName: nfs
      nfs:
        path: /nfs/data/03
        # nfs 主节点服务器的 IP
        server: 139.198.36.162

vi pv.yaml
    
    # 复制上面文件
    
    kubectl apply -f pv.yaml
    
    # 查看 pv， kubectl get pv
    kubectl get persistentvolume

![在这里插入图片描述][ebcc809d8756474c8c211e55d84375f2.png]

###### 7.3.2 创建、绑定 PCV ######

相当于创建设用pv的申请书

创建pvc

pvc.yaml

apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: nginx-pvc
    spec:
      accessModes:
        - ReadWriteMany
      resources:
        requests:
          # 需要 200M的 PV
          storage: 200Mi
      # 上面 PV 写的什么 这里就写什么    
      storageClassName: nfs

vi pvc.yaml
    
    # 复制上面配置
    
    kubectl get pv
    
    kubectl apply -f pvc.yaml
    
    kubectl get pv
    
    kubectl get pvc

![在这里插入图片描述][c2db73f8b9f843e1bcd3860edcc2ad5c.png]  
绑定了， 绑定了1G的，10M 不够，3G太大，就选择了 1G  
![在这里插入图片描述][af7792cc5de34341a2822ef59e7fa289.png]

###### 7.3.3 创建 Pod 绑定 PVC ######

创建 Pod，绑定 PVC

apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: nginx-deploy-pvc
      name: nginx-deploy-pvc
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: nginx-deploy-pvc
      template:
        metadata:
          labels:
            app: nginx-deploy-pvc
        spec:
          containers:
          - image: nginx
            name: nginx
            volumeMounts:
            - name: html
              mountPath: /usr/share/nginx/html
          volumes:
            - name: html
              # 之前是 nfs，这里用 pvc
              persistentVolumeClaim:
                claimName: nginx-pvc

vi dep02.yaml
    
    # 复制上面 yaml
    
    kubectl apply -f dep02.yaml
    
    kubectl get pod
    
    kubectl get pv
    
    kubectl get pvc

![在这里插入图片描述][c99f18e9743e4df6a8451b970ef6e24a.png]  
挂载后，测试  
![在这里插入图片描述][b14e9d9f95404d5995602c9058bafb41.png]  
![在这里插入图片描述][b1e6b18dffde4541ba20070a7bfb726a.png]  
进入 Pod 内部查看 同步的文件  
![在这里插入图片描述][a7c6b2804b1d4495bfc63907f24cf73b.png]  
![在这里插入图片描述][34217cf4da0e4583a43c06c6d8986357.png]

##### 7.4 ConfigMap ★ #####

ConfigMap：抽取应用配置，并且可以自动更新。挂载配置文件， PV 和 PVC 是挂载目录的。

###### 7.4.1 redis示例 ######

1.  创建 ConfigMap  
    创建 / 删除 cm

vi redis.conf
    # 写
    appendonly yes
    
    # 创建配置，redis保存到k8s的etcd；
    kubectl create cm redis-conf --from-file=redis.conf
    
    # 查看
    kubectl get cm
    
    rm -rf redis.conf

![在这里插入图片描述][3b5b5d009cae4b88a0321e6a961fdaa6.png]

# 查看 ConfigMap 的 yaml 配置咋写的
    kubectl get cm redis-conf -oyaml

![在这里插入图片描述][69fa2ab40f3c47dd8d2327816849ac0e.png]

apiVersion: v1
    data:    # data是所有真正的数据，key：默认是文件名   value：配置文件的内容（appendonly yes 是随便写的）
      redis.conf: |
        appendonly yes
    kind: ConfigMap
    metadata:
      name: redis-conf
      namespace: default

1.  创建 Pod  
    redis.yaml

apiVersion: v1
    kind: Pod
    metadata:
      name: redis
    spec:
      containers:
      - name: redis
        image: redis
        command:
          # 启动命令
          - redis-server
          # 指的是redis容器内部的位置
          - "/redis-master/redis.conf"  
        ports:
        - containerPort: 6379
        volumeMounts:
        - mountPath: /data
          name: data
        - mountPath: /redis-master
          name: config
      volumes:
        - name: data
          emptyDir: {
        }
        - name: config
          configMap:
            name: redis-conf
            items:
            - key: redis.conf
              path: redis.conf

redis.conf 会放在 /redis-master 下  
![在这里插入图片描述][b39280a3d2c24b83ba1251fe47820668.png]

vi redis.yaml
    
    # 复制上面配置
    
    kubectl apply -f redis.yaml
    
    kubectl get pod

![在这里插入图片描述][87b9d7fc3ec74863932187499ff44f7d.png]  
页面中 进入刚才创建的 pod redis 内部  
![在这里插入图片描述][8eb57d64546e4b9f9e34c747e805ff3f.png]  
查看 redis.conf 配置文件 内容  
![在这里插入图片描述][8a4981bbcc4c4eab823725c1fa30a870.png]

kubectl get cm
    
    # 修改配置 里 redis.conf 的内容
    kubectl edit cm redis-conf

修改 redis-conf 的 redis.conf 内容  
![在这里插入图片描述][723746d94b5a4ec19fdd71f8cfb145b8.png]  
修改 redis-conf 的 redis.conf 内容

过了一会， 就同步了  
![在这里插入图片描述][b1b9ba7f0bb24d409354ede48cb1d68f.png]

![在这里插入图片描述][48667889bc1947afb3acaf1f048273ad.png]  
3. 检查默认配置

kubectl exec -it redis -- redis-cli
    
    127.0.0.1:6379> CONFIG GET appendonly
    127.0.0.1:6379> CONFIG GET requirepass

![在这里插入图片描述][ca9e46d7bf344b74a0827b3bd86ee20a.png]  
和 命令行一样的  
![在这里插入图片描述][2216aa8e7c15411592b4ad4ac0b7a624.png]  
删除，重新创建 Pod，更新 配置文件的 配置值  
![在这里插入图片描述][6cbabb99fb5d43ce83b2770a5e3d4a41.png]  
查看 更新的 配置值  
![在这里插入图片描述][e5f57eb8f41d4c56ae4b254e82a4f76d.png]

总结：  
● 修改了 ConfigMap，Pod里面的配置文件会跟着同步。  
● 但配置值 未更改，需要重新启动 Pod 才能从关联的ConfigMap 中获取 更新的值。 Pod 部署的中间件 自己本身没有热更新能力。

##### 7.5 Secret #####

Secret ：是对象类型，用来保存敏感信息，例如密码、OAuth 令牌和 SSH 密钥。 将这些信息放在 secret 中比放在 Pod 的定义或者 容器镜像 中来说更加安全和灵活。

###### 7.5.1 拉取失败 ######

Docker hub 仓库中，自己的仓库设置成私有的。 然后去 下载私有的。下载不了（未登录）。

自己的仓库设置成私有的![在这里插入图片描述][3058a93f0903449fbbfc80d2acf60a7f.png]  
查看 拉取命令  
![在这里插入图片描述][8258c6a5b44f4e8096fdbb1672b5d5a9.png]  
拒绝拉取  
![在这里插入图片描述][b160f4fbc504425192a6daa9f65b2fdd.png]  
mypod.yaml

apiVersion: v1
    kind: Pod
    metadata:
      name: private-cgxin-docker
    spec:
      containers:
      - name: private-cgxin-docker
        image: cgxin/cgxin_docker:1.0

vi mypod.yaml
    
    # 复制上面配置
    
    kubectl apply -f mypod.yaml
    
    kubectl get pod

![在这里插入图片描述][795d47dca62946e69c5a67fa3875aa91.png]  
提示 镜像拉取失败

可视化界面 查看错误描述：也是没有权限。  
![在这里插入图片描述][c3f5a925a26f4f34baf2f742a31bd9da.png]  
删除配置文件 创建的错误 Pod  
![在这里插入图片描述][c6a84621e7914be78f32544b55ba6fa3.png]

###### 7.5.2 创建 Secret ######

创建 Secret

kubectl create secret docker-registry cgxin-docker-secret \
    --docker-username=leifengyang \
    --docker-password=Lfy123456 \
    --docker-email=534096094@qq.com
    
    ##命令格式
    kubectl create secret docker-registry regcred \
      --docker-server=<你的镜像仓库服务器> \
      --docker-username=<你的用户名> \
      --docker-password=<你的密码> \
      --docker-email=<你的邮箱地址>

# 查看
    kubectl get secret
    
    kubectl get secret cgxin-docker-secret -oyaml

![在这里插入图片描述][bdf32a97b8c741baba3d53894c2de5c9.png]  
重新修改 配置文件，加入 Secret

apiVersion: v1
    kind: Pod
    metadata:
      name: private-cgxin-docker
    spec:
      containers:
      - name: private-cgxin-docker
        image: cgxin/cgxin_docker:1.0
      # 加上 Secret  
      imagePullSecrets:
      - name: cgxin-docker-secret

vi mypod.yaml
    
    # 复制上面配置
    
    kubectl apply -f mypod.yaml
    
    kubectl get pod

使用 Secret 后，可以成功 拉取下来了。  
![在这里插入图片描述][f5575c160f084d36aa317c95919e817e.png]

Docker Hub 镜像 复原成 public  
![在这里插入图片描述][e6fc5ee8b94b476ab7fc1720aaca7c42.png]

总结：  
可视化界面 操作很方便  
![在这里插入图片描述][68a49ed6fa60463c84da36acc5db36af.png]

[https_kubernetes.github.io_ingress-nginx]: https://kubernetes.github.io/ingress-nginx/
[c6c9c7b12d0b424ab5177fbb5f5ba904.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/181113cfb3e7480692b959b2c0241805.png
[a239b51524a044eab3a44f783b3dc738.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/79e7512575d54308bd29d8d6e77b1ea5.png
[90998cb8b1e5495f8446a5792d5b45c7.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/c43bee4049de4ed38acd5966c6daeae0.png
[daa733d8980e45e8aa1c3d1bd2d02a9d.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/fcb455341ee94e25afd93b8165f2a11c.png
[06125fc4d9034ca8b15c97fec1f7c11a.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/14889080e68d4e1f9d96bc87b5fc13b2.png
[55deb9e1808b4ae69518759f439dae4f.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/6903e9631a204f2bbff408ee0bcc2f70.png
[3d1f079fd0f34f258548c23e35b8441f.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/8e73b14c7ff3472cadf1dfe2ba95ba6b.png
[e989b076c0704cb388dbf660fdf01443.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/9e6e79d9d7394a498866b564149fb2d6.png
[3c6626272d0e4137a21752e8c06b9fd4.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/186054874c9042ca9a30bea8b2cb7f68.png
[62a68d78314b48a6a7bcc24b6b4f8f94.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/2557a2af076c49dfa67432e4f2237260.png
[d17b8e0a8dd542e79e5bd88180519957.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/d3b1a44f6cd2479ca6e89274aec848d6.png
[1ddc6ecf6df645e3915db7ef12f17aa7.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/7aa060e6d58444928ade32231c52d8e6.png
[1edb2ae49ff740e5949b0e120dc22bff.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/dd2b3229bcb3498aa6350e8a0f04a73d.png
[5868c7642c1c46cc8173fac994669347.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/e39e228557e14866bc593f9b8d02440c.png
[543120835d1f40b7ae38e0883f83d9cd.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/bacfb78b64ec48a88724d2a1f8de07e2.png
[1b34d2824980464fa8703caded69b6b3.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/e3a6e03326034f1396ed865059855269.png
[3b4b76aa07264e029a511f2624144268.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/383c373b7d644df1a0ca4cca006c9d41.png
[631cad721260446f87e5f0a1d95560e1.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/4227ddd15d3346eda6b913a217928b62.png
[ee2d73b33e8d424789e7cfa615ca0563.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/c367f3037f0c4e87822edf6dddc79fbe.png
[e736170a2c4d423e90a8b2b5c1035330.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/ceda3d2cde4b4406852306f3dfd0416e.png
[151afef67e6643329f27d28af04bf516.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/8c3bb1d3fb3a4e7297fc75f156f0443e.png
[a190c8623f7f4116b869b27bc875234d.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/0660bba02f4d4266a13f2558898715f9.png
[e78e7e49256049d18fe461668e7903a2.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/b6f73bee7f6943818aa3ac61f1aa98c3.png
[64640333f8aa4de7bd2ff3c0d46eb083.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/a6f5db3d03b24f32a41940530e51a7d3.png
[adcfbb51429b45c1ae4b5494ef432017.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/57774d93c1db4619a2411381896a50ab.png
[5c2801184c99416b9854ed3918659f81.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/7ef9171c686b4efcad885a6343a18f19.png
[d7b526fbff4243a8b06a550c7948ded3.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/1f959cdabc2c49d0a774d82343d623ed.png
[859b510207774be28caf80c2bc608e84.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/fc20b86b0f6349ef8983af54e77d21a2.png
[024ae8a6727949939fd7276d91a75736.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/38345b82d1e846e4bc4ffa41bfbe0727.png
[42a56bc45fc6469296504f6158f06bbf.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/2af14743e89a4e468435fb4fc9ea9ce3.png
[8ffd9a0a9ef141958005745fc5cb0e51.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/72f58670b09949c6afc2215ed7535ff7.png
[https_kubernetes.github.io_ingress-nginx_user-guide_nginx-configuration]: https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration
[f16530ae471140268efb1e7878711ac0.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/e8b010440b5748ca9840a9d8271697c1.png
[522a07b833854c3fb8dc9e40c267500e.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/d7a6245094774410806198b5fb53f4f1.png
[757a4f0566244948b083c461cf87fcb7.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/07109919e99d4515a4f8fa05f23a47a4.png
[fa900ebb41194383bb908f915ae4300e.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/7fd091d7e2ca41b69577bf1ceb7c1e33.png
[917ad66897e542bba7ea73873bb61ba4.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/a1917b07dc34445cae7c521ab969198c.png
[32b9d080ef08458d8f45882a33e4fcae.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/d7a8d532f2824f2c89c7e7b68bc38b86.png
[21d970037120440fa2ab3ae2429e0e46.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/d45a52e6c047480990afbdea07b60f8f.png
[517f221647994c8e828891133837c452.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/9b31816e0fc74665a540f51e943f23cf.png
[0b2d84df440d44feabe3e30b2e9e1da9.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/20f5f3577e724755828585d8710c3f0c.png
[047f9b7a618b4e9aa50dfb7321a2aada.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/51ca2fb7a08343568c6906b47ca44eec.png
[b9e65882ad16415083143e401934ed30.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/055af82c5f344934b43cca815833aaf3.png
[f495f79f03d846d5b66cc846fe4e1e56.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/4f00673512c94104aa8415dce27c2511.png
[05e5e564c75f406bb82bf00266f0ec39.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/397acfec5d27436e8b6c57949c13a55b.png
[a60b78ca6b7c4890981acb8bc68e01f3.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/f89f8097f8ce43168d827661825832c4.png
[ebcc809d8756474c8c211e55d84375f2.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/d05738ee66dd4245b0341d0aeb37b9c2.png
[c2db73f8b9f843e1bcd3860edcc2ad5c.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/3c4ec3d11f9d46a9818565a450f8210e.png
[af7792cc5de34341a2822ef59e7fa289.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/0e3b6ee30a124eb3ae2f7d3934a2f700.png
[c99f18e9743e4df6a8451b970ef6e24a.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/ac8b7426c9734e44bf2ebbd5458c8892.png
[b14e9d9f95404d5995602c9058bafb41.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/c9b7ed2992324ad19d33c371e1d31172.png
[b1e6b18dffde4541ba20070a7bfb726a.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/fd0d7b52b805464181b4a7b6e1b895d7.png
[a7c6b2804b1d4495bfc63907f24cf73b.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/e8ecff17dd1a43c48f14c0909febf3ba.png
[34217cf4da0e4583a43c06c6d8986357.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/8d8e5d23138b419fbbc7ef5d956d134a.png
[3b5b5d009cae4b88a0321e6a961fdaa6.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/41e69ef7594742798c903f74777930c2.png
[69fa2ab40f3c47dd8d2327816849ac0e.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/ef94b5e868e04cdd9dc6294765c9544d.png
[b39280a3d2c24b83ba1251fe47820668.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/16b08d367ba6492a87f12aeab298cec4.png
[87b9d7fc3ec74863932187499ff44f7d.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/d12cf587765c49fe97b9cb5f18c96ea4.png
[8eb57d64546e4b9f9e34c747e805ff3f.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/f95700c481654694961637c8a71f1fe9.png
[8a4981bbcc4c4eab823725c1fa30a870.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/706afd6275a14be0ad4e468f04c6b598.png
[723746d94b5a4ec19fdd71f8cfb145b8.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/3b241b047af84f3c891633b8d4682d4d.png
[b1b9ba7f0bb24d409354ede48cb1d68f.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/d8291cf17006492ca42ee09e04475fcf.png
[48667889bc1947afb3acaf1f048273ad.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/af90407ad88b48c2bb80c0440455be01.png
[ca9e46d7bf344b74a0827b3bd86ee20a.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/07876e186ccf48d088f84816a3d5a79c.png
[2216aa8e7c15411592b4ad4ac0b7a624.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/d5fe1a3c404d42549a3dadc0ba8c79a6.png
[6cbabb99fb5d43ce83b2770a5e3d4a41.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/c653fea11e5c4d01a1f3fc4272e2ff96.png
[e5f57eb8f41d4c56ae4b254e82a4f76d.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/5d2ed167d5744a7e8496effc4c488fb2.png
[3058a93f0903449fbbfc80d2acf60a7f.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/45ff7993dce94a48a76a38a183fc8caa.png
[8258c6a5b44f4e8096fdbb1672b5d5a9.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/ca6676b5c9db480f81012fddd878caa7.png
[b160f4fbc504425192a6daa9f65b2fdd.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/125e0654115f46d895abf852eb9680cf.png
[795d47dca62946e69c5a67fa3875aa91.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/6a911ac69b064e5a9b11ac0c407b8e0d.png
[c3f5a925a26f4f34baf2f742a31bd9da.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/5a16be491e944e9cbe399d2b30c02670.png
[c6a84621e7914be78f32544b55ba6fa3.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/4a04c6496e9146219367f22dc730cdd0.png
[bdf32a97b8c741baba3d53894c2de5c9.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/276ec460d25041eb82f539f4de7c14ae.png
[f5575c160f084d36aa317c95919e817e.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/45cd773581b749d883cd70998fca51c1.png
[e6fc5ee8b94b476ab7fc1720aaca7c42.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/ef46243931494cc0be5c93041fa61379.png
[68a49ed6fa60463c84da36acc5db36af.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/30/6323fdf2bbd143be836533283310e44c.png