[kubernetes]step2-基础服务ingress-nginx并添加https证书

电玩女神 2023-07-01 13:55 7阅读 0赞

# 基础服务ingress-nginx #

接着上面阿里云上的kubernetes集群，一步步搭建服务

### 创建一个hostNetwork网络模式的ingress-nginx ###

##### 暴露hostport #####

hostport 比桥接模式 效率高 并且没有网络转发，在任何一个节点都可以访问。nodeport会有网络转发，因为不知道在哪个节点，会多一次转发。

##### 首先需要添加labels #####

kubectl label node k8s-op-n03 app=ingress

mandatory.yaml

---
    
    apiVersion: v1
    kind: Namespace
    metadata:
      name: ingress-nginx
    ---
    
    apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
      name: default-http-backend
      labels:
        app.kubernetes.io/name: default-http-backend
        app.kubernetes.io/part-of: ingress-nginx
      namespace: ingress-nginx
    spec:
      replicas: 1
      selector:
        matchLabels:
          app.kubernetes.io/name: default-http-backend
      template:
        metadata:
          labels:
            app.kubernetes.io/name: default-http-backend
            app.kubernetes.io/part-of: ingress-nginx
        spec:
          terminationGracePeriodSeconds: 60
          containers:
          - name: default-http-backend
            # Any image is permissible as long as:
            # 1. It serves a 404 page at /
            # 2. It serves 200 on a /healthz endpoint
            image: gcr.io/google_containers/defaultbackend:1.4
            livenessProbe:
              httpGet:
                path: /healthz
                port: 8080
                scheme: HTTP
              initialDelaySeconds: 30
              timeoutSeconds: 5
            ports:
            - containerPort: 8080
            resources:
              limits:
                cpu: 10m
                memory: 20Mi
              requests:
                cpu: 10m
                memory: 20Mi
    ---
    
    apiVersion: v1
    kind: Service
    metadata:
      name: default-http-backend
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: default-http-backend
        app.kubernetes.io/part-of: ingress-nginx
    spec:
      ports:
      - port: 80
        targetPort: 8080
      selector:
        app.kubernetes.io/name: default-http-backend
    ---
    
    kind: ConfigMap
    apiVersion: v1
    metadata:
      name: nginx-configuration
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    ---
    
    kind: ConfigMap
    apiVersion: v1
    metadata:
      name: tcp-services
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    ---
    
    kind: ConfigMap
    apiVersion: v1
    metadata:
      name: udp-services
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    ---
    
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: nginx-ingress-serviceaccount
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    
    ---
    
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: ClusterRole
    metadata:
      name: nginx-ingress-clusterrole
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    rules:
      - apiGroups:
          - ""
        resources:
          - configmaps
          - endpoints
          - nodes
          - pods
          - secrets
        verbs:
          - list
          - watch
      - apiGroups:
          - ""
        resources:
          - nodes
        verbs:
          - get
      - apiGroups:
          - ""
        resources:
          - services
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - "extensions"
        resources:
          - ingresses
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - ""
        resources:
            - events
        verbs:
            - create
            - patch
      - apiGroups:
          - "extensions"
        resources:
          - ingresses/status
        verbs:
          - update
    
    ---
    
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: Role
    metadata:
      name: nginx-ingress-role
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    rules:
      - apiGroups:
          - ""
        resources:
          - configmaps
          - pods
          - secrets
          - namespaces
        verbs:
          - get
      - apiGroups:
          - ""
        resources:
          - configmaps
        resourceNames:
          # Defaults to "<election-id>-<ingress-class>"
          # Here: "<ingress-controller-leader>-<nginx>"
          # This has to be adapted if you change either parameter
          # when launching the nginx-ingress-controller.
          - "ingress-controller-leader-nginx"
        verbs:
          - get
          - update
      - apiGroups:
          - ""
        resources:
          - configmaps
        verbs:
          - create
      - apiGroups:
          - ""
        resources:
          - endpoints
        verbs:
          - get
    
    ---
    
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: RoleBinding
    metadata:
      name: nginx-ingress-role-nisa-binding
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: Role
      name: nginx-ingress-role
    subjects:
      - kind: ServiceAccount
        name: nginx-ingress-serviceaccount
        namespace: ingress-nginx
    
    ---
    
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: ClusterRoleBinding
    metadata:
      name: nginx-ingress-clusterrole-nisa-binding
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: nginx-ingress-clusterrole
    subjects:
      - kind: ServiceAccount
        name: nginx-ingress-serviceaccount
        namespace: ingress-nginx
    ---
    
    apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
      name: nginx-ingress-controller
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    spec:
      replicas: 1
      selector:
        matchLabels:
          app.kubernetes.io/name: ingress-nginx
      template:
        metadata:
          labels:
            app.kubernetes.io/name: ingress-nginx
          annotations:
            prometheus.io/port: '10254'
            prometheus.io/scrape: 'true'
        spec:
          serviceAccountName: nginx-ingress-serviceaccount
    # hostNetwork网络
          hostNetwork: true
    # 节点选择器 标签app=nginx
          nodeSelector:
            app: ingress
          containers:
            - name: nginx-ingress-controller
              image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.19.0
              args:
                - /nginx-ingress-controller
                - --default-backend-service=$(POD_NAMESPACE)/default-http-backend
                - --configmap=$(POD_NAMESPACE)/nginx-configuration
                - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
                - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
                - --publish-service=$(POD_NAMESPACE)/ingress-nginx
                - --annotations-prefix=nginx.ingress.kubernetes.io
              securityContext:
                capabilities:
                    drop:
                    - ALL
                    add:
                    - NET_BIND_SERVICE
                # www-data -> 33
                runAsUser: 33
              env:
                - name: POD_NAME
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.name
                - name: POD_NAMESPACE
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.namespace
              ports:
              - name: http
                containerPort: 80
              - name: https
                containerPort: 443
              livenessProbe:
                failureThreshold: 3
                httpGet:
                  path: /healthz
                  port: 10254
                  scheme: HTTP
                initialDelaySeconds: 10
                periodSeconds: 10
                successThreshold: 1
                timeoutSeconds: 1
              readinessProbe:
                failureThreshold: 3
                httpGet:
                  path: /healthz
                  port: 10254
                  scheme: HTTP
                periodSeconds: 10
                successThreshold: 1
                timeoutSeconds: 1

创建ingress-nginx 服务

kubectl apply -f mandatory.yaml

如果一直处于下载镜像或者镜像下载失败的话 可以到node上执行

azk8spull  gcr.io/google_containers/defaultbackend:1.4
    azk8spull  quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.19.0

如果pod日志报错0/10 nodes are available: 10 node(s) didn’t match node selector.

![image-20200120164746549][]

kubectl label node k8s-op-n03 app=ingress

加上标签之后就会进入containercreating状态

![image-20200120165052563][]

全部ok了

![image-20200120170811675][]

![image-20200120165221071][]

在k8s-op-n03上看到80和443都已经被ingress-nginx启动的nginx pod 占用了

![image-20200120170636076][]

#####  #####

##### 创建一个用于存放证书的文件夹 #####

mkdir -p /root/nginx-crt/

##### 上传证书到/root/nginx-crt/ 创建secret 保存https证书 #####

kubectl create secret tls aircourses --cert=2349945__aircourses.com.pem --key=2349945__aircourses.com.key

##### 查看证书 #####

kubectl get secret

![image-20200120180115188][]

##### 创建一个nginx-demo.yaml 并给k8s-op-n03添加dns解析为k8s-tomcat-demo.aircourses.com #####

如果没有解析也可以修改本机的/etc/hosts

#deploy
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: tomcat-demo
    spec:
      selector:
        matchLabels:
          app: tomcat-demo
      replicas: 1
      template:
        metadata:
          labels:
            app: tomcat-demo
        spec:
          containers:
          - name: tomcat-demo
            image: registry.cn-hangzhou.aliyuncs.com/liuyi01/tomcat:8.0.51-alpine
            ports:
            - containerPort: 8080
    ---
    #service
    apiVersion: v1
    kind: Service
    metadata:
      name: tomcat-demo
    spec:
      ports:
      - port: 80
        protocol: TCP
        targetPort: 8080
      selector:
        app: tomcat-demo
    
    ---
    #ingress
    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
      name: tomcat-demo
    spec:
      rules:
      - host: k8s-tomcat-demo.aircourses.com
        http:
          paths:
          - path: /
            backend:
              serviceName: tomcat-demo
              servicePort: 80

kubectl apply -f nginx-demo.yaml

![image-20200120172150821][]

##### 创建nginx-demo-tls.yaml #####

![image-20200121102004160][]

##### 验证完成之后 删除 #####

值得注意的是

部署 TLS 后 80 端口也会自动重定向到 443

Ingress 开启TLS后，是http>https , https>https

[image-20200120164746549]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nL2ltYWdlLTIwMjAwMTIwMTY0NzQ2NTQ5LnBuZw?x-oss-process=image/format,png
[image-20200120165052563]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nL2ltYWdlLTIwMjAwMTIwMTY1MDUyNTYzLnBuZw?x-oss-process=image/format,png
[image-20200120170811675]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nL2ltYWdlLTIwMjAwMTIwMTcwODExNjc1LnBuZw?x-oss-process=image/format,png
[image-20200120165221071]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nL2ltYWdlLTIwMjAwMTIwMTY1MjIxMDcxLnBuZw?x-oss-process=image/format,png
[image-20200120170636076]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nL2ltYWdlLTIwMjAwMTIwMTcwNjM2MDc2LnBuZw?x-oss-process=image/format,png
[image-20200120180115188]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nL2ltYWdlLTIwMjAwMTIwMTgwMTE1MTg4LnBuZw?x-oss-process=image/format,png
[image-20200120172150821]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nL2ltYWdlLTIwMjAwMTIwMTcyMTUwODIxLnBuZw?x-oss-process=image/format,png
[image-20200121102004160]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nL2ltYWdlLTIwMjAwMTIxMTAyMDA0MTYwLnBuZw?x-oss-process=image/format,png