k8s (二十六) --- kubernetes Helm之通过kubeapps使用图形部署helm应用

川长思鸟来 2023-03-13 11:48 44阅读 0赞

### kubernetes Helm之通过kubeapps使用图形部署helm应用 ###

*  一、部署kubeapps
 *  二、kubeapps的登陆
 *  三、kubeapps的使用
 *   *  添加自己的chart库
     *  使用chart部署ingress-nginx应用
     *  更新应用
     *  应用回滚
     *  应用删除

kubeapps应用可以为Helm提供web UI界面管理。

# 一、部署kubeapps #

首先需要添加一个第三方库：

[root@server1 helm]# helm repo add bitnami https://charts.bitnami.com/bitnami
    "bitnami" has been added to your repositories
    [root@server1 helm]# 
    [root@server1 helm]# helm search repo kubeapps
    NAME            	CHART VERSION	APP VERSION	DESCRIPTION                                       
    bitnami/kubeapps	3.6.0        	v1.10.0    	Kubeapps is a dashboard for your Kubernetes clu...

最新的版本是3.6.0，我们使用3.4.3，拉取解压部署文件：

[root@server1 helm]# helm pull bitnami/kubeapps --version 3.4.3
    [root@server1 helm]# ls
    kubeapps-3.4.3.tgz  nfs-client-provisioner            nginx-ingress             pvc.yaml
    mychart             nfs-client-provisioner-1.2.8.tgz  nginx-ingress-1.36.3.tgz
    [root@server1 helm]# tar zxf kubeapps-3.4.3.tgz 
    [root@server1 helm]# ls
    kubeapps            mychart                 nfs-client-provisioner-1.2.8.tgz  nginx-ingress-1.36.3.tgz
    kubeapps-3.4.3.tgz  nfs-client-provisioner  nginx-ingress                     pvc.yaml
    [root@server1 helm]# cd kubeapps/
    [root@server1 kubeapps]# ls
    charts  Chart.yaml  crds  README.md  requirements.lock  requirements.yaml  templates  values.schema.json  values.yaml

可以发现对比以前的应用多了一个`requirements.yaml`文件，这个文件定义了应用的依赖性：

[root@server1 kubeapps]# cat requirements.yaml 
    dependencies:
      - name: mongodb
        version: ">= 0"
        repository: https://kubernetes-charts.storage.googleapis.com
        condition: mongodb.enabled
      - name: postgresql
        version: ">= 0"
        repository: https://kubernetes-charts.storage.googleapis.com
        condition: postgresql.enabled

可以看出其依赖mongodb以及postgresql。

部署kubeapps需要的镜像：

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70]可以在变量文件中查看镜像的版本，为了方便这里新建了一个harbor项目bitnami（文件中的镜像就是这个地址）。

编辑变量文件：

[root@server1 kubeapps]# vim values.yaml

默认镜像仓库为docker.io我们需要在全局指定我们的镜像仓库：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 1]使用helm v3版本：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 2]  
打开ingress并指定主机名：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 3]

定义初始化仓库：![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 4]其他的我们不做更改。

安装：

[root@server1 kubeapps]# kubectl create namespace kubeapps
    namespace/kubeapps created
    
    [root@server1 kubeapps]# helm -n kubeapps install kubeapps .

等待一会查看状态：

[root@server1 kubeapps]# kubectl -n kubeapps get all
    NAME                                                             READY   STATUS      RESTARTS   AGE
    pod/apprepo-kubeapps-sync-bitnami-1589314200-j8w4d               0/1     Completed   0          28m
    pod/apprepo-kubeapps-sync-bitnami-1589314800-6rcsc               0/1     Completed   0          18m
    pod/apprepo-kubeapps-sync-bitnami-1589315400-86k5k               0/1     Completed   4          8m2s
    pod/apprepo-kubeapps-sync-bitnami-nzpwn-7nt47                    0/1     Completed   2          115m
    pod/apprepo-kubeapps-sync-stable-1589314200-j5h4j                0/1     Completed   0          28m
    pod/apprepo-kubeapps-sync-stable-1589314800-xnsnm                0/1     Completed   0          18m
    pod/apprepo-kubeapps-sync-stable-1589315400-n79lb                0/1     Completed   0          8m2s
    pod/apprepo-kubeapps-sync-stable-f2m2x-7bsfb                     0/1     Completed   0          115m
    pod/kubeapps-6688575df9-cqt2m                                    1/1     Running     0          116m
    pod/kubeapps-6688575df9-j7qx6                                    1/1     Running     0          116m
    pod/kubeapps-internal-apprepository-controller-8cdf54665-d7fgv   1/1     Running     0          116m
    pod/kubeapps-internal-assetsvc-6b94785ffb-6rwzm                  1/1     Running     3          116m
    pod/kubeapps-internal-assetsvc-6b94785ffb-vvk9l                  1/1     Running     2          116m
    pod/kubeapps-internal-dashboard-6b65678647-54vc4                 1/1     Running     0          116m
    pod/kubeapps-internal-dashboard-6b65678647-dpkms                 1/1     Running     0          116m
    pod/kubeapps-internal-kubeops-59887fdbf8-hgqpr                   1/1     Running     0          116m
    pod/kubeapps-internal-kubeops-59887fdbf8-mm9x8                   1/1     Running     0          116m
    pod/kubeapps-mongodb-685d4c95d9-lmr2h                            1/1     Running     0          116m
    
    NAME                                  TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)     AGE
    service/kubeapps                      ClusterIP   10.104.41.207   <none>        80/TCP      116m
    service/kubeapps-internal-assetsvc    ClusterIP   10.107.221.83   <none>        8080/TCP    116m
    service/kubeapps-internal-dashboard   ClusterIP   10.102.5.115    <none>        8080/TCP    116m
    service/kubeapps-internal-kubeops     ClusterIP   10.102.76.222   <none>        8080/TCP    117m
    service/kubeapps-mongodb              ClusterIP   10.104.39.53    <none>        27017/TCP   116m
    
    NAME                                                         READY   UP-TO-DATE   AVAILABLE   AGE
    deployment.apps/kubeapps                                     2/2     2            2           116m
    deployment.apps/kubeapps-internal-apprepository-controller   1/1     1            1           116m
    deployment.apps/kubeapps-internal-assetsvc                   2/2     2            2           116m
    deployment.apps/kubeapps-internal-dashboard                  2/2     2            2           116m
    deployment.apps/kubeapps-internal-kubeops                    2/2     2            2           116m
    deployment.apps/kubeapps-mongodb                             1/1     1            1           116m
    
    NAME                                                                   DESIRED   CURRENT   READY   AGE
    replicaset.apps/kubeapps-6688575df9                                    2         2         2       116m
    replicaset.apps/kubeapps-internal-apprepository-controller-8cdf54665   1         1         1       116m
    replicaset.apps/kubeapps-internal-assetsvc-6b94785ffb                  2         2         2       116m
    replicaset.apps/kubeapps-internal-dashboard-6b65678647                 2         2         2       116m
    replicaset.apps/kubeapps-internal-kubeops-59887fdbf8                   2         2         2       116m
    replicaset.apps/kubeapps-mongodb-685d4c95d9                            1         1         1       116m
    
    NAME                                                 COMPLETIONS   DURATION   AGE
    job.batch/apprepo-kubeapps-sync-bitnami-1589314200   1/1           14s        28m
    job.batch/apprepo-kubeapps-sync-bitnami-1589314800   1/1           17s        18m
    job.batch/apprepo-kubeapps-sync-bitnami-1589315400   1/1           2m31s      8m2s
    job.batch/apprepo-kubeapps-sync-bitnami-nzpwn        1/1           6m         115m
    job.batch/apprepo-kubeapps-sync-stable-1589314200    1/1           27s        28m
    job.batch/apprepo-kubeapps-sync-stable-1589314800    1/1           26s        18m
    job.batch/apprepo-kubeapps-sync-stable-1589315400    1/1           22s        8m2s
    job.batch/apprepo-kubeapps-sync-stable-f2m2x         1/1           3m43s      115m
    
    NAME                                          SCHEDULE       SUSPEND   ACTIVE   LAST SCHEDULE   AGE
    cronjob.batch/apprepo-kubeapps-sync-bitnami   */10 * * * *   False     0        8m9s            115m
    cronjob.batch/apprepo-kubeapps-sync-stable    */10 * * * *   False     0        8m9s            115m

可以看出创建了两个cronjob，每隔10分钟同步我们定义的两个仓库。

部署成功

# 二、kubeapps的登陆 #

首先需要做解析：

[root@foundation63 ~]# cat /etc/hosts
    ......
    172.25.63.3 kubeapps.redhat.org www1.westos.org www2.westos.org www3.westos.org

浏览器访问`kubeapps.redhat.org`：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 5]需要token登陆，因此我们需要创建sa并为其附加`cluster-admin`的权限：

[root@server1 kubeapps]# kubectl create serviceaccount kubeapps-operator -n kubeapps 
    serviceaccount/kubeapps-operator created
    [root@server1 kubeapps]# 
    [root@server1 kubeapps]# kubectl -n kubeapps get sa
    NAME                                         SECRETS   AGE
    default                                      1         122m
    kubeapps-internal-apprepository-controller   1         122m
    kubeapps-internal-kubeops                    1         122m
    kubeapps-operator                            1         6s
    [root@server1 kubeapps]# 
    [root@server1 kubeapps]# kubectl create clusterrolebinding kubeapps-operator --clusterrole=cluster-admin --serviceaccount=kubeapps:kubeapps-operator
    clusterrolebinding.rbac.authorization.k8s.io/kubeapps-operator created

查看token：

[root@server1 kubeapps]# kubectl -n kubeapps describe sa kubeapps-operator 
    Name:                kubeapps-operator
    Namespace:           kubeapps
    Labels:              <none>
    Annotations:         <none>
    Image pull secrets:  <none>
    Mountable secrets:   kubeapps-operator-token-5gb84
    Tokens:              kubeapps-operator-token-5gb84
    Events:              <none>
    [root@server1 kubeapps]# kubectl -n kubeapps describe secrets kubeapps-operator-token-5gb84
    Name:         kubeapps-operator-token-5gb84
    Namespace:    kubeapps
    Labels:       <none>
    Annotations:  kubernetes.io/service-account.name: kubeapps-operator
                  kubernetes.io/service-account.uid: aa9f9695-e250-4f89-962b-225e375a198d
    
    Type:  kubernetes.io/service-account-token
    
    Data
    ====
    ca.crt:     1025 bytes
    namespace:  8 bytes
    token:      eyJhbGciOiJSUzI1NiIsImtpZCI6Ilp5SmtWcG42LUZiMGhaR3Rac3dUT01HQ0RkdFpvaE00ZkNGNnJuend6dmMifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlYXBwcyIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJlYXBwcy1vcGVyYXRvci10b2tlbi01Z2I4NCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJlYXBwcy1vcGVyYXRvciIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImFhOWY5Njk1LWUyNTAtNGY4OS05NjJiLTIyNWUzNzVhMTk4ZCIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlYXBwczprdWJlYXBwcy1vcGVyYXRvciJ9.ecyGNwfTqXS7JyhNFXqjfS0iYZT5lbwE0iRETJHOqNF75w1DzVD88RNQ8ofgWqqQtPTJXDCyEkSmKaRR0re55DAifdvmx5qtySN07j7sjznsoNk1doKKWRMjV_zBzNGfB-XPL1JGgDPP2pTRR6_sBnpG2mhS6RhJ3fcVjgJF_B10LXw6Fhi-HeyAMhY9i4hAtWb6sDOkPGPp4NTJTQhlgDWRg7HHGrfW-v-gsvUzIl8Rn-sD3NXR6NRhuvenCdyTMbetVRrZrRa_tdSDjA6WA4c8qgVs_y5rROLsRb0-K1FEOnWV937GA8VCrS0mq-BCzGUP94_Z6DfgNMvxuHb4ig

复制token到网页端登陆：![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 6]登陆成功。

# 三、kubeapps的使用 #

查看所有的namespace：

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 7]上面展示的就是我们之前部署过的应用。

## 添加自己的chart库 ##

因为我们仓库的地址为`reg.westos.org`，在pod内不能解析这个地址，因此我们首先需要添加pod内解析：

[root@server1 kubeapps]# kubectl -n kube-system get cm
    NAME                                 DATA   AGE
    coredns                              1      25d
    extension-apiserver-authentication   6      25d
    kube-flannel-cfg                     2      25d
    kube-proxy                           2      25d
    kubeadm-config                       2      25d
    kubelet-config-1.18                  1      25d
    [root@server1 kubeapps]# kubectl -n kube-system edit cm coredns
    configmap/coredns edited

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 8]之后网页端添加仓库：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 9]![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 10]  
信息填写完成后安装。  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 11]

安装后会进行同步。

[root@server1 kubeapps]# kubectl -n kubeapps get pod
    NAME                                                         READY   STATUS              RESTARTS   AGE
    ......
    apprepo-kubeapps-sync-mychart-5xqzz-mh72q                    0/1     Completed           0          49s
    apprepo-kubeapps-sync-mychart-c6m9f-qlgk9                    0/1     ContainerCreating   0          27s
    apprepo-kubeapps-sync-mychart-q5kbv-kngz4                    0/1     Completed           0          30s
    ......

同步完成后会显示Completed。

在网页端就可以搜到：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 12]点进入可以看到两个版本0.1.0和0.2.0：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 13]

## 使用chart部署ingress-nginx应用 ##

点击Deploy：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 14]更改部署文件，使用ingress：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 15]  
更改后提交，等待一会变成ready：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 16]测试访问：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 17]

部署成功。

## 更新应用 ##

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 18]  
点击Upgrade,更改版本号：

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 19]更改后点击提交：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 20]再次访问发现应用已更新：

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 21]

## 应用回滚 ##

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 22]

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 23]等待ready后，再次测试访问：

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 24]可以看出已经回滚到了v1.

## 应用删除 ##

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 25]

可以看出通过kubeapps图形部署应用极大的简化了我们部署helm应用的过程。

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70]: /images/20230312/84dca2569ddb4d6f9591e4c767ea3713.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 1]: /images/20230312/fdb3baf1682d4826b71c207c0067a13d.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 2]: /images/20230312/2eeda54357a74be6aff1afd9de0fd5ea.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 3]: https://img-blog.csdnimg.cn/20200513132734607.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 4]: https://img-blog.csdnimg.cn/2020051313293248.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 5]: https://img-blog.csdnimg.cn/20200513133426296.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 6]: https://img-blog.csdnimg.cn/20200513133736403.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 7]: https://img-blog.csdnimg.cn/20200513133813984.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 8]: https://img-blog.csdnimg.cn/20200513134259704.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 9]: https://img-blog.csdnimg.cn/20200513134035361.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 10]: https://img-blog.csdnimg.cn/20200513134645614.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 11]: https://img-blog.csdnimg.cn/20200513134919188.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 12]: https://img-blog.csdnimg.cn/20200513134946917.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 13]: https://img-blog.csdnimg.cn/20200513135025971.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 14]: https://img-blog.csdnimg.cn/20200513135253125.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 15]: https://img-blog.csdnimg.cn/20200513135309347.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 16]: https://img-blog.csdnimg.cn/20200513135522124.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 17]: https://img-blog.csdnimg.cn/20200513135545259.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 18]: https://img-blog.csdnimg.cn/20200513135650391.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 19]: https://img-blog.csdnimg.cn/202005131357295.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 20]: https://img-blog.csdnimg.cn/20200513135821755.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 21]: https://img-blog.csdnimg.cn/20200513135844743.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 22]: https://img-blog.csdnimg.cn/20200513135958168.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 23]: https://img-blog.csdnimg.cn/20200513140048592.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 24]: https://img-blog.csdnimg.cn/20200513140132448.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2_size_16_color_FFFFFF_t_70 25]: https://img-blog.csdnimg.cn/20200513140214498.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1ODg3NTQ2,size_16,color_FFFFFF,t_70