Jenkis pipeline构建项目实践-编写podTemplate实现和k8s对接

迈不过友情╰ 2022-11-29 13:30 540阅读 0赞

## 微信公众号搜索 DevOps和k8s全栈技术 ，即可关注公众号，也可扫描文章最后的二维码关注公众号，每天会分享技术文章供大家阅读参考哈~ ##

> 在完成前文的jenkins server 在k8s环境部署之后，本文我们来测试在k8s集群环境中的jenkins pipeline构建项目和更新，具体环境要求如下：  
> 1、jenkins pipeline插件安装成功  
> 2、要更新的应用已提前部署  
> 3、Jenkins slave中需要有kubectl、svn、mvn客户端且环境变量设置准确  
> 4、Jenkis slave需要能和master的api-server进行正常通信（这里为了简便，前文构建jenkins server的时候直接对defalut这个service account做rabc授权。其他可选方案是使用~/.kube/config文件实现客户端的授权认证）  
> 5、需要配置共享卷(slave编译完成之后把war包copy到共享卷位置)

# 一、创建构建任务 #

1、点击“jenkins ”——“新建 ”——“pipeline”，输入项目的名称，点击“OK”  
![format_png][]

2、下拉到“pipeline”配置，填写脚本  
![format_png 1][]  
具体脚本内容如下：

podTemplate(name: 'jenkins-slave', cloud: 'kubernetes',
      namespace: 'default', label: 'jenkins-slave',
      serviceAccount: 'default', containers: [
      containerTemplate(
          name: 'jenkins-slave',
          image: 'harbor.59iedu.com/fjhb/jenkins-slave-toolkit:2018-08-10-v1',
          args: '${computer.jnlpmac} ${computer.name}',
          ttyEnabled: true,
          privileged: false,
          alwaysPullImage:true,
          )
      ],
      volumes: [
        persistentVolumeClaim(mountPath: '/tmp/', claimName: 'tomcat-jcsj-data')
      ]) {
      node('jenkins-slave') {
        stage('svn-checkout') {
          container('jnlp') {
              sh """
              svn checkout --username=yanglw --password=mypassword http://192.168.1.81/svn/fjhbjsb/k8s-pipeline-test --non-interactive 
              """
          }
        }
    
        stage('mvn-package') {
          container('jnlp') {
              sh """
              mkdir -p /var/jenkins_home && cd k8s-pipeline-test && mvn clean package && cp -rpf target/*.war /tmp/
              """
          }
        }
    
        stage('restart') {
          container('jnlp') {
              sh """
              pod_name=`kubectl  get pods -l name=jcsj-dev -o name | cut -d"/" -f2`
              kubectl delete pod \$pod_name
              """
          }
        }
    
        }
    }

3、脚本释义  
podTemplate节指定创建pod的模板和环境

> Name：为pod的名称前缀  
> Cloud：为构建pod的云环境，需要和前面新建的云环境名称一样  
> Namespace：创建pod所在的namespace  
> Label: 创建pod对应的标签  
> serviceAccount：pod使用sa，这里使用default, 所以前文创建jenkins master deployment的时候多创建了一个default-role,这样自动创建出来的的jenkins-slave具有对应的api权限

containerTemplate节

> 指定创建容器的模板，当云环境里面配置pod模板后，容器模板以云环境的配置为准。

Volumes节

> 配置jenkins-slave pod挂载的卷，当云环境里面配置pod模板后，挂载卷以云环境配置为准

stage节为具体的pipeline步骤

> 这里第一步进行svn代码迁出；  
> 第二步进行编译，并把包传到共享卷上面；  
> 第三步对pod进行删除，因为应用采用deployment方式部署，所以可以实现重新创建pod，达到更新的效果。

# 二、构建准备 #

在开始构建之前，有必要介绍一下jcsj-dev的环境，这个环境是需要提前部署好的。部署文件如下

# cat deploy.yml 
    ---
    apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
      name: jcsj-dev
      namespace: default
    spec:
      replicas: 1
      template:
        metadata:
          labels:
            name: jcsj-dev
        spec:
          containers:
          - name: jcsj-dev
            image: tomcat:latest
            imagePullPolicy: IfNotPresent
            ports:
            - name: web
              containerPort: 8080
            volumeMounts:
            - mountPath: /usr/local/tomcat/webapps
              name: tomcat-jcsj-data
          volumes:
          - name: tomcat-jcsj-data
            persistentVolumeClaim:
              claimName: tomcat-jcsj-data
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: jcsj-dev
    spec:
      type: NodePort
      ports:
        - port: 8080
          targetPort: 8080
          nodePort: 8453
      selector:
    name: jcsj-dev

# cat pv.yaml 
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: tomcat-jcsj-data
    spec:
      capacity:
        storage: 5Gi 
      accessModes:
      - ReadWriteMany 
      nfs: 
        path: /home/tomcat_jcsj
        server: 192.168.115.6
      persistentVolumeReclaimPolicy: Recycle 
    
    ---
    kind: PersistentVolumeClaim
    apiVersion: v1
    metadata:
      name: tomcat-jcsj-data
    spec:
      accessModes:
        - ReadWriteMany
      resources:
        requests:
          storage: 5Gi

![format_png 2][]  
![format_png 3][]

# 三、更新项目代码 #

更新index.jsp并提交  
![format_png 4][]  
![format_png 5][]

# 四、开始构建 #

点击“立即构建”，构建过程中可以通过控制台输出看具体的日志信息  
![format_png 6][]  
![format_png 7][]  
![format_png 8][]

构建过程中观察jenkins master的日志输出：

# kubectl logs -f jenkins-master-588b89c75f-ztvgm 
    INFO: Started provisioning Kubernetes Pod Template from kubernetes with 1 executors. Remaining excess workload: 0
    Aug 13, 2018 10:14:22 AM hudson.slaves.NodeProvisioner$2 run
    INFO: Kubernetes Pod Template provisioning successfully completed. We have now 2 computer(s)
    Aug 13, 2018 10:14:22 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Created Pod: jenkins-slave-gjjbh in namespace default
    Aug 13, 2018 10:14:22 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for Pod to be scheduled (0/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:28 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Container is waiting jenkins-slave-gjjbh [jnlp]: ContainerStateWaiting(message=null, reason=ContainerCreating, additionalProperties={})
    Aug 13, 2018 10:14:28 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for Pod to be scheduled (1/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:34 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for agent to connect (1/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:35 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for agent to connect (2/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:36 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for agent to connect (3/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:37 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for agent to connect (4/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:38 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for agent to connect (5/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:39 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for agent to connect (6/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:40 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for agent to connect (7/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:41 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for agent to connect (8/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:42 AM hudson.TcpSlaveAgentListener$ConnectionHandler run
    INFO: Accepted JNLP4-connect connection #1 from /172.30.66.4:47408
    Aug 13, 2018 10:14:42 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for agent to connect (9/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:43 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for agent to connect (10/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:44 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for agent to connect (11/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:45 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for agent to connect (12/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:46 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for agent to connect (13/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:47 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for agent to connect (14/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:48 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for agent to connect (15/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:49 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for agent to connect (16/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:50 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for agent to connect (17/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:51 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for agent to connect (18/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:52 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for agent to connect (19/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:14:53 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
    INFO: Waiting for agent to connect (20/100): jenkins-slave-gjjbh
    Aug 13, 2018 10:15:12 AM org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1 doLaunch
    INFO: Created process inside pod: [jenkins-slave-gjjbh], container: [jnlp] with pid:[-1]
    Aug 13, 2018 10:15:21 AM org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1 doLaunch
    INFO: Created process inside pod: [jenkins-slave-gjjbh], container: [jnlp] with pid:[-1]
    Aug 13, 2018 10:16:04 AM org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1 doLaunch
    INFO: Created process inside pod: [jenkins-slave-gjjbh], container: [jnlp] with pid:[-1]
    Aug 13, 2018 10:16:06 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
    INFO: Terminating Kubernetes instance for agent jenkins-slave-gjjbh
    Aug 13, 2018 10:16:06 AM jenkins.slaves.DefaultJnlpSlaveReceiver channelClosed
    WARNING: Computer.threadPoolForRemoting [#8] for jenkins-slave-gjjbh terminated
    java.nio.channels.ClosedChannelException
            at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:208)
            at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222)
            at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832)
            at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287)
            at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:181)
            at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:283)
            at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:503)
            at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:248)
            at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:200)
            at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:213)
            at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:800)
            at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:173)
            at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:311)
            at hudson.remoting.Channel.close(Channel.java:1295)
            at hudson.remoting.Channel.close(Channel.java:1263)
            at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:708)
            at hudson.slaves.SlaveComputer.access$800(SlaveComputer.java:96)
            at hudson.slaves.SlaveComputer$3.run(SlaveComputer.java:626)
            at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
            at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
            at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            at java.lang.Thread.run(Thread.java:748)
    
    Aug 13, 2018 10:16:06 AM org.csanchez.jenkins.plugins.kubernetes.pipeline.PodTemplateStepExecution$PodTemplateCallback finished
    INFO: Removing pod template and deleting pod jenkins-slave-7gnwx from cloud kubernetes
    Aug 13, 2018 10:16:06 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
    INFO: Terminated Kubernetes instance for agent default/jenkins-slave-gjjbh
    Aug 13, 2018 10:16:06 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
    INFO: Disconnected computer jenkins-slave-gjjbh
    Terminated Kubernetes instance for agent default/jenkins-slave-gjjbh
    Aug 13, 2018 10:16:06 AM org.csanchez.jenkins.plugins.kubernetes.pipeline.PodTemplateStepExecution$PodTemplateCallback finished
    WARNING: Failed to delete pod for agent default/jenkins-slave-7gnwx: not found
    Aug 13, 2018 10:16:07 AM org.jenkinsci.plugins.workflow.job.WorkflowRun finish
    INFO: k8s-pipeline-test #1 completed: SUCCESS

# 五、验证结果 #

![format_png 9][]

# 六、后记 #

本文仅介绍的pipeline方式构建项目的过程，实际应用中还需要解决若干问题，例如：

> 1、svn checkout步骤需要明文配置用户名和密码，存在安全性问题  
> 2、 每次执行构建都需要全量迁出代码，效率不高  
> 3、要更新的应用需要提前部署好，不能适配首次编译部署场景  
> 4、对共享卷存在依赖，解耦问题需要解决

# 七、续 #

研究了pipeline的语法结合shell脚本，基本可以解决上述问题，以下是pipeline代码

podTemplate(name: 'jenkins-slave', cloud: 'kubernetes',
      namespace: 'default', label: 'jenkins-slave') 
      {
      node('jenkins-slave') {
        stage('svn-checkout') {
          container('jnlp') {
             checkout([$class: 'SubversionSCM', 
             additionalCredentials: [], 
             excludedCommitMessages: '', 
             excludedRegions: '', 
             excludedRevprop: '', 
             excludedUsers: '', 
             filterChangelog: false, 
             ignoreDirPropChanges: false, 
             includedRegions: '', 
             locations: [[cancelProcessOnExternalsFail: true, 
             credentialsId: 'ef21f2d3-cb39-468a-aa19-9175d96a365f', 
             depthOption: 'infinity', 
             ignoreExternalsOption: true, 
             local: '.', 
             remote: 'http://192.168.1.81/svn/fjhbjsb/k8s-pipeline-test']], 
             quietOperation: true, workspaceUpdater: [$class: 'UpdateUpdater']])
          }
        }
    
        stage('mvn-package') {
          container('jnlp') {
              sh """
              mvn clean package 
              """
          }
        }
    
        stage('build-docker-image') {
          container('jnlp') {
            sh '''
            IMAGE_NAME="harbor.59iedu.com/fjhb/jcsj:v$BUILD_NUMBER"   
            CONATINER_NAME="jcsj-dev" 
            echo "192.168.1.227 harbor.59iedu.com" >> /etc/hosts 
            sed -i  "s/BUILD_NUMBER/$BUILD_NUMBER/g" tomcat.yaml
            sed -i  "s/BUILD_NUMBER=1/BUILD_NUMBER=$BUILD_NUMBER/g" rolling-update.sh 
            docker login harbor.59iedu.com -u admin -p Harbor12345 
            docker build -t $IMAGE_NAME . 
            docker push $IMAGE_NAME
            '''
    
          }
        }
    
        stage('apply update') {
          container('jnlp') {
              sh '''
              NAMESPACE=default
              DEPLOYMENT=tomcat-jcsj
              REGISTRY=harbor.59iedu.com/fjhb/jcsj:v$BUILD_NUMBER
              FILE=tomcat.yaml       
              STATUS=$(kubectl get deployment -l k8s-app=$DEPLOYMENT -o name -n $NAMESPACE)
              if [ ! $STATUS ];then
              kubectl create -f $FILE
              else
              kubectl set image deployment/$DEPLOYMENT $DEPLOYMENT=$REGISTRY --namespace=$NAMESPACE
              fi    
              '''
          }
        }
        }
    }

其中，svn-checkout节代码通过“流水线语法”工具生成，同时需要给default sa用户相应的权限,否则无法实现apply update节配置

# kubectl create clusterrolebinding k8s-admin --clusterrole=cluster-admin  --user=system:serviceaccount:default:default

![format_png 10][]  
![format_png 11][]  
![format_png 12][]  
![format_png 13][]

文章转载地址如下:

https://blog.51cto.com/ylw6006/2159861

往期精彩文章

[kubernetes全栈技术+企业案例演示【带你快速掌握和使用k8s】][kubernetes_k8s]

[突破运维和开发瓶颈、Python、k8s、DevOps转型一网打尽！][Python_k8s_DevOps]

[python运维开发实战-基础篇  
][python_-_]

[python运维和开发实战-高级篇][python_-]

[python运维和开发实战-安装和创建Django项目][python_-_Django]

[Docker公司禁止被列入美国"实体名单"的国家、企业使用][Docker]

[kubernetes面试题汇总][kubernetes]

[DevOps视频和资料免费领取][DevOps]

[kubernetes技术分享-可用于企业内部培训][kubernetes_-]

[谈谈我的IT发展之路][IT]

[kubernetes系列文章第一篇-k8s基本介绍][kubernetes_-k8s]

[kubernetes系列文章第二篇-kubectl][kubernetes_-kubectl]

[了解pod和pod的生命周期-这一篇文章就够了  
][pod_pod_-_]

[Kubernetes中部署MySQL高可用集群][Kubernetes_MySQL]

[Prometheus+Grafana+Alertmanager搭建全方位的监控告警系统-超详细文档][Prometheus_Grafana_Alertmanager_-]

[k8s1.18多master节点高可用集群安装-超详细中文官方文档][k8s1.18_master_-]

[k8s中蓝绿部署、金丝雀发布、滚动更新汇总][k8s]

[运维常见问题汇总-tomcat篇][-tomcat]

[关于linux内核参数的调优，你需要知道][linux]

[kubernetes挂载ceph rbd和cephfs][kubernetes_ceph rbd_cephfs]

[报警神器Alertmanager发送报警到多个渠道][Alertmanager]

[jenkins+kubernetes+harbor+gitlab构建企业级devops平台][jenkins_kubernetes_harbor_gitlab_devops]

[kubernetes网络插件-flannel篇][kubernetes_-flannel]

[kubernetes网络插件-calico篇][kubernetes_-calico]

[kubernetes认证、授权、准入控制][kubernetes 1]

[限制不同的用户操作k8s资源][k8s 1]

[面试真题&技术资料免费领取-覆盖面超全~][-]

[Prometheus监控MySQL][Prometheus_MySQL]

[Prometheus监控Nginx][Prometheus_Nginx]

[Prometheus监控Tomcat][Prometheus_Tomcat]

[linux面试题汇总][linux 1]

[测试通过storageclass动态生成pv][storageclass_pv]

[通过编写k8s的资源清单yaml文件部署gitlab服务][k8s_yaml_gitlab]

[helm安装和使用-通过helm部署k8s应用][helm_-_helm_k8s]

[k8s基于Ingress-nginx实现灰度发布][k8s_Ingress-nginx]

[k8s的Pod安全策略][k8s_Pod]

[Prometheus Operator-上篇-安装和使用篇][Prometheus Operator-_-]

[Prometheus Operator-下篇  
][Prometheus Operator-_]

[通过kubeconfig登陆k8s的dashboard ui界面][kubeconfig_k8s_dashboard ui]

[通过token令牌登陆k8s dashboard ui界面][token_k8s dashboard ui]

[kubernetes集群的etcd数据库详细介绍][kubernetes_etcd]

[Linux网络流量监控工具][Linux]

[kubernetes搭建EFK日志管理系统][kubernetes_EFK]

[prometheus operator监控k8s集群之外的haproxy组件][prometheus operator_k8s_haproxy]

[kubernetes ConfigMap存储卷][kubernetes ConfigMap]

## 技术交流群    ##

学无止境，了解更多关于kubernetes/docker/devops/openstack/openshift/linux/IaaS/PaaS相关内容，想要获取更多资料和免费视频，可按如下方式进入技术交流群

**扫码加群????**

![format_png 14][]

**微信：**luckylucky421302

## 微信公众号    ##

**                                    ** **长按指纹关注公众号????**

![format_png 15][]

**                                       点击在看少个 bug????**

[format_png]: /images/20221124/9ac4dd7124c641e1bbb42969767e5ece.png
[format_png 1]: /images/20221124/cd0525118f4c456c9f4d1ca5c9f2eea3.png
[format_png 2]: /images/20221124/66e07a442cf4488a8379047070d52a94.png
[format_png 3]: /images/20221124/37f043678b21403e91c62326e9ebcee0.png
[format_png 4]: /images/20221124/a4b46103cd4e48e0ad4fd06c722ac9b8.png
[format_png 5]: /images/20221124/d5a8cf24607541ceabd224478024563a.png
[format_png 6]: /images/20221124/303a4daa30324952a7e1be6724da6eb6.png
[format_png 7]: /images/20221124/405f0f8f6e6f437db2dd405754c08c27.png
[format_png 8]: /images/20221124/cbe1b7a28bfb4302a660d7d858f1ff40.png
[format_png 9]: /images/20221124/ede0414612c1434e9d560831d596a287.png
[format_png 10]: /images/20221124/498f59eec3244147a86deb1b95f89fd0.png
[format_png 11]: /images/20221124/06f7ff6f99b3432eb6b8d29beeaa668b.png
[format_png 12]: /images/20221124/976ba25f5a5b4e7a97bc1c3a031aef73.png
[format_png 13]: /images/20221124/830dd01c8ec14f6cb67340b96bda9e77.png
[kubernetes_k8s]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638a31cc14032772d778f1a8fa5dc73f9aba6c99546c9ab25d9f91f706be3cff34572f0c1a&idx=1&mid=2247484708&scene=21&sn=2a7230ade0ce0bd9059e92752e03ba18#wechat_redirect
[Python_k8s_DevOps]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638455cc140d43dd091b09efc0c744b82f32d905c78ce9be71aae8c6d324d625f110fa1093&idx=1&mid=2247486272&scene=21&sn=e993bc259db899b79922863859d24bb2#wechat_redirect
[python_-_]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb63855acc140c4cf0346cc3433bc874e7cd37a5b17cf5254147a1b20b538cf70158f2234b0f&idx=1&mid=2247486031&scene=21&sn=58003addbf36f2858f7d0dfbf887bb2a#wechat_redirect
[python_-]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb6385eccc140cfa90bef6759c768c94ac2ca904c827fb2dcd969858ba46b818145e76e09c62&idx=1&mid=2247486201&scene=21&sn=0d45a2660a7480d73cf72ba67fe52f3a#wechat_redirect
[python_-_Django]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638417cc140d011a058d7cf83795bf75cfc327acbb3727c968058ff2807874a26f87843080&idx=1&mid=2247486210&scene=21&sn=078e172a34c22e5db1f57b17c21b2efe#wechat_redirect
[Docker]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb63841ecc140d08e72aa244da69b776d4470aac79f3d03610c5fec827ac24a2048178204bf0&idx=1&mid=2247486219&scene=21&sn=31fe60b84f327ac1cc24bdb98ccba6a6#wechat_redirect
[kubernetes]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638b9ccc14028a3220533e5e05f81b5b434cdc14f1cb4be6771a8afdb943f7f5f940704c61&idx=1&mid=2247484553&scene=21&sn=141700cdd2e13b87d5b756f8965a30c2#wechat_redirect
[DevOps]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638b81cc140297e292675f1e8ea21d3a9f05e01e488e642bcc1113d69d06d0afbd40446c0e&idx=1&mid=2247484564&scene=21&sn=3ec4b58056aec30acf82e84cfd9daa52#wechat_redirect
[kubernetes_-]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638b4acc14025c10c5b87f75ff8e95c89c97036f1f73250a64e2a829df191c95c34ed5b8b3&idx=1&mid=2247484511&scene=21&sn=7900c5a4eac3535be97315797d42d64b#wechat_redirect
[IT]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638c29cc14053fafcb62be244f5a279052b8d159c9c530e73495cfd8ab97205988780d4aaa&idx=1&mid=2247484220&scene=21&sn=72ae437836108fbecbeb86752aad888c#wechat_redirect
[kubernetes_-k8s]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638c42cc1405547ea583833c46878b6c120f5f39fb3746f89a8d2e515d432422b3019ded64&idx=1&mid=2247484247&scene=21&sn=c8986ed491e6c48cb6faab087fe8472c#wechat_redirect
[kubernetes_-kubectl]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638cbacc1405ac390b6c0fc7535e1cc805877a408c5f4a94a7148a47c186a57ff4f0c8ea32&idx=3&mid=2247484335&scene=21&sn=7e5ae586dae2016f6c0f5fcb60dd7317#wechat_redirect
[pod_pod_-_]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638c30cc1405263cbd23e3fbe23f9ca9e83c1eb1800216a0e4e26e7c27d4234598def46bb1&idx=1&mid=2247484197&scene=21&sn=36ece95b266609bd769cf6a1fcf9d723#wechat_redirect
[Kubernetes_MySQL]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638c73cc1405657ed02b3126423259cb8563c18d268fa7ffd4a516327cd35148ef61d8d762&idx=1&mid=2247484262&scene=21&sn=de970288426cdca05cf6ca0f5df4ff4c#wechat_redirect
[Prometheus_Grafana_Alertmanager_-]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638b08cc14021e5e025f2f0b89d18ec5ef19f9dec4429a56f356e8b596d01095e313dab3e3&idx=1&mid=2247484445&scene=21&sn=bf8928e7aae815b8e7f9ac56a67289ce#wechat_redirect
[k8s1.18_master_-]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638c7dcc14056b9f0261e05e4d36a057c4511fb5806222f71b7965c3573fed86672b2c93b7&idx=1&mid=2247484264&scene=21&sn=4d74de0cb117c07ba7c73ea21c42bf94#wechat_redirect
[k8s]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638c36cc140520e6905db5923afe163d7babb5d9eb6c5e8045a795c37b33a2a2e5541e3efd&idx=1&mid=2247484195&scene=21&sn=b841f2ea305acfa2996a667d4ff4d99e#wechat_redirect
[-tomcat]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638c21cc140537d10eeb8b4eb37e1451392123dd74513d704af00506c0576ba11da9907735&idx=1&mid=2247484212&scene=21&sn=923b044e9d2b4e5a612ac7c1307d166c#wechat_redirect
[linux]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638c5ecc14054862495a8f77d58d7483c34310839ea3ed93d8b44875a1685b3412d381f23a&idx=1&mid=2247484235&scene=21&sn=12e2e80187dbf217efff6c59da778f7c#wechat_redirect
[kubernetes_ceph rbd_cephfs]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638a07cc140311e6e111b9bb89e3613c495a26b6b496fe97c838787568c72dd1bb23088520&idx=1&mid=2247484690&scene=21&sn=32deb33cef9d91518cd8dea47da073c5#wechat_redirect
[Alertmanager]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638ab7cc1403a1f6b07cbe682548595fe683f1ffdf56d2b4d31382003907a41da1e03c1408&idx=1&mid=2247484834&scene=21&sn=7f407f5a27b03b4757c8892f1b5d6234#wechat_redirect
[jenkins_kubernetes_harbor_gitlab_devops]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638d2fcc14043990967471ba8a2aaf389ab483594fd81395b5afda1c23a9eff4371df6bca4&idx=1&mid=2247483962&scene=21&sn=595eac335f7fb523bbb60cbbede109ca#wechat_redirect
[kubernetes_-flannel]: http://mp.weixin.qq.com/s?__biz=MzIzOTQ3NDg1Mw%3D%3D&chksm=e928c005de5f4913b3e81d32cf98dc4f237be04b0c19e02f615650538a9aa0aabdc025604918&idx=1&mid=2247483855&scene=21&sn=26587a359cca53d129ab7e8a7fa394e0#wechat_redirect
[kubernetes_-calico]: http://mp.weixin.qq.com/s?__biz=MzIzOTQ3NDg1Mw%3D%3D&chksm=e928c073de5f49658fdaed40affb1a9ea3f059e7df3cc90ae24b194a8698a7f73089d3964cab&idx=1&mid=2247483833&scene=21&sn=c00c2b69db7d06eacaf6206805d2955a#wechat_redirect
[kubernetes 1]: http://mp.weixin.qq.com/s?__biz=MzIzOTQ3NDg1Mw%3D%3D&chksm=e928c05fde5f494917570591168d9e71f507c2d124312c51dfdb6cfb30eb03eb48b73a3e0d9b&idx=1&mid=2247483797&scene=21&sn=bc208ee075c41b81ffabaefe509538ba#wechat_redirect
[k8s 1]: http://mp.weixin.qq.com/s?__biz=MzIzOTQ3NDg1Mw%3D%3D&chksm=e928c0b4de5f49a2eef8cc91afa81095b71eb093e94dce6eddbc7059583e819b1ac51c596ebb&idx=1&mid=2247483774&scene=21&sn=dfc3babde045fe99da9fa168cc9f0595#wechat_redirect
[-]: http://mp.weixin.qq.com/s?__biz=MzIzOTQ3NDg1Mw%3D%3D&chksm=e928c0bfde5f49a9990c672e4434f5ae7b74004dc1982cc44801c524b58067d9b245e5ba5b71&idx=1&mid=2247483765&scene=21&sn=b8444e82a93535425349b509ded4e4b6#wechat_redirect
[Prometheus_MySQL]: http://mp.weixin.qq.com/s?__biz=MzIzOTQ3NDg1Mw%3D%3D&chksm=e928c0aade5f49bc7df29ecb2280ab9e1a8aa905b31e36856bf8dc19b05ef60c5e7be0333b88&idx=1&mid=2247483744&scene=21&sn=1035d73f8cf39caf8534766435497590#wechat_redirect
[Prometheus_Nginx]: http://mp.weixin.qq.com/s?__biz=MzIzOTQ3NDg1Mw%3D%3D&chksm=e928c095de5f4983c249b1b556fa6f8f60a0bb6991d2872079a9d5c28ce2ee17b8fc407eeb2c&idx=1&mid=2247483743&scene=21&sn=3ad15cd693685479dc7bf5dd628796d2#wechat_redirect
[Prometheus_Tomcat]: http://mp.weixin.qq.com/s?__biz=MzIzOTQ3NDg1Mw%3D%3D&chksm=e928c086de5f499011eb542c6da8ead32b538c0bf06340ebe56ad439e2649e0b63efbcb4a611&idx=1&mid=2247483724&scene=21&sn=dd390f05270c6ac7a9c2b51fdc22f0ee#wechat_redirect
[linux 1]: http://mp.weixin.qq.com/s?__biz=MzIzOTQ3NDg1Mw%3D%3D&chksm=e928c0dede5f49c878898e04329f08d8adf554397476283aaa3f2554f71a8bf20986755fbcc9&idx=1&mid=2247483668&scene=21&sn=a7d0768b7eb034ef2e2f23e25b1b1845#wechat_redirect
[storageclass_pv]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638837cc140121578456f6a5d679ae481421f2f3f1d5099bb7f2c56d00174fe61b2016061d&idx=1&mid=2247485218&scene=21&sn=ba698d444a029a2659091bfd81133697#wechat_redirect
[k8s_yaml_gitlab]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638814cc1401028c7ab321d2ac6579f5c6d2b029fadc5687874e41470bf95aa09c01c33d45&idx=1&mid=2247485185&scene=21&sn=2b5646119a008d69948c5a84d3724c95#wechat_redirect
[helm_-_helm_k8s]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638bbbcc1402ad16d4b65b3db26cbd324c85819934077caa41969f0c9d573b71b21e2c51ff&idx=1&mid=2247484590&scene=21&sn=2c23098269c5df1e793aa47198a7b1e6#wechat_redirect
[k8s_Ingress-nginx]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb6385fdcc140ceb4a5cf902d24499ce3480f91aa97e0e0c7a1fb12a466865ff2f8dedc8be0c&idx=1&mid=2247486184&scene=21&sn=cccb2bc67ca74a01be7b4f79e05f8bec#wechat_redirect
[k8s_Pod]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb6385c3cc140cd57c41dcdd267f2bc0f0f6476d3a56e1fa88b23a7ad3f82799bad72cbdde93&idx=1&mid=2247486166&scene=21&sn=a40899007709d1a1f3e7b5be79ed6e49#wechat_redirect
[Prometheus Operator-_-]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638626cc140f30c8d35e45239e6b559f7369b8c58bacd157a24e7b62313c826baec75f299f&idx=2&mid=2247485747&scene=21&sn=3da14a91d75f745b26c3aa8d10361d23#wechat_redirect
[Prometheus Operator-_]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb63865bcc140f4d8966d1c3745c8f69c6384337c43a1a8eda6e1ae488bd78f08e44c9553b19&idx=1&mid=2247485774&scene=21&sn=1b0301dadc4b737e19b1066219813db7#wechat_redirect
[kubeconfig_k8s_dashboard ui]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638679cc140f6f4e024f2e7739b995fc235b6f4e97983a583bc9c7e82da88cf0fbf8736fe7&idx=1&mid=2247485804&scene=21&sn=c157f0596f130bd97ec745b278546c50#wechat_redirect
[token_k8s dashboard ui]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638687cc140f918f04ca60ed188da26cd3ab37a79e6fdbf64531efd11ae60dac0a77788f7c&idx=2&mid=2247485842&scene=21&sn=163174142218e8dca3bb9a94ed173694#wechat_redirect
[kubernetes_etcd]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb63853bcc140c2d4c4da1b59a4865c193383392d92075cee80530866892d8a2a365e2b4e565&idx=1&mid=2247485998&scene=21&sn=9889882d1820ac005e5512eccb67c0f1#wechat_redirect
[Linux]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb638538cc140c2e581b8c673a9c529359042a2262ec112f8c3476c7408f25e04856f0b37bbd&idx=2&mid=2247485997&scene=21&sn=ac694beaf60cbf18d133eb6ebbc7a345#wechat_redirect
[kubernetes_EFK]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb63851fcc140c09d17fe6e270524da6c3ec3c5f3969f6f4f61a54c54724168801d93b2cda9d&idx=1&mid=2247485962&scene=21&sn=87dfde054f2ff489a1ffd28dfe02a435#wechat_redirect
[prometheus operator_k8s_haproxy]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb6386f0cc140fe6dea923cb07c2164db1c894c87d8c85d828aa269cdf4ae2d92eeb201adb67&idx=1&mid=2247485925&scene=21&sn=5ae9bc969963775e347c3a1f279d541c#wechat_redirect
[kubernetes ConfigMap]: http://mp.weixin.qq.com/s?__biz=MzU0NjEwMTg4Mg%3D%3D&chksm=fb63853acc140c2ca1dc3ca813b133ca7158c34df5cfe59cebf7621851ead91490eaabc74a1b&idx=1&mid=2247485999&scene=21&sn=f6b03745426149e498a8c7065b060e24#wechat_redirect
[format_png 14]: /images/20221124/7ade65985ae741adada4383901c7604a.png
[format_png 15]: /images/20221124/1bf8297a80f1454fad1fd424eff8e1d6.png