kubernetes学习:3.创建tls证书和秘钥

墨蓝 2022-05-28 07:41 260阅读 0赞

# kubernete tls证书和秘钥 #

传输k8s各组件之间的通信可以使用http方式，但是为了安全起见，生产环境需要使用https方式通信，所以我们需要生成tls证书进行加密传输。

--------------------

## 构建环境 ##

再次介绍下我们本次搭建的环境：

<table> 
 <thead> 
  <tr> 
   <th>节点名称</th> 
   <th>ip</th> 
   <th>配置</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>wecloud-test-k8s-1（master）</td> 
   <td>192.168.99.183</td> 
   <td>4核，4G，50G磁盘</td> 
  </tr> 
  <tr> 
   <td>wecloud-test-k8s-2（node1）</td> 
   <td>192.168.99.189</td> 
   <td>4核，4G，50G磁盘</td> 
  </tr> 
  <tr> 
   <td>wecloud-test-k8s-3（node2）</td> 
   <td>192.168.99.185</td> 
   <td>4核，4G，50G磁盘</td> 
  </tr> 
  <tr> 
   <td>wecloud-test-k8s-4（node3）</td> 
   <td>192.168.99.196</td> 
   <td>4核，4G，50G磁盘</td> 
  </tr> 
 </tbody> 
</table>

生成tls证书的工具:cfssl

**生成的 CA 证书和秘钥文件如下**：  
ca-key.pem  
ca.pem  
kubernetes-key.pem  
kubernetes.pem  
kube-proxy.pem  
kube-proxy-key.pem  
admin.pem  
admin-key.pem

**使用证书的组件如下**：

etcd：使用 ca.pem、kubernetes-key.pem、kubernetes.pem；  
kube-apiserver：使用 ca.pem、kubernetes-key.pem、kubernetes.pem；  
kubelet：使用 ca.pem；  
kube-proxy：使用 ca.pem、kube-proxy-key.pem、kube-proxy.pem；  
kubectl：使用 ca.pem、admin-key.pem、admin.pem；  
kube-controller-manager：使用 ca-key.pem、ca.pem

证书的创建过程是在192.168.99.183（master）节点上进行的，生成好证书只需要把证书分发给其他节点，这样就可以进行加密通信了。

--------------------

## 安装tls和秘钥 ##

--------------------

### 安装cfssl ###

直接在官网下载cfssl相关二进制包：

wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
    wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
    wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64

增加可执行权限后，将可执行二进制文件拷贝到全局变量可以识别的目录/usr/local/bin:

[root@wecloud-test-k8s-1 ~]# chmod +x cfssl-certinfo_linux-amd64 
    [root@wecloud-test-k8s-1 ~]# chmod +x cfssljson_linux-amd64 
    [root@wecloud-test-k8s-1 ~]# chmod +x cfssl_linux-amd64 
    [root@wecloud-test-k8s-1 ~]# mv cfssl_linux-amd64 /usr/local/bin/cfssl
    [root@wecloud-test-k8s-1 ~]# mv cfssljson_linux-amd64 /usr/local/bin/cfssljson
    [root@wecloud-test-k8s-1 ~]# mv cfssl-certinfo_linux-amd64 /usr/local/bin/cfssl-certinfo

### 创建CA文件 ###

--------------------

#### 创建 CA 配置文件 ####

[root@wecloud-test-k8s-1 ~]# mkdir /root/ssl
    [root@wecloud-test-k8s-1 ~]# cd /root/ssl/
    [root@wecloud-test-k8s-1 ssl]# cfssl print-defaults config > config.json
    [root@wecloud-test-k8s-1 ssl]# cfssl print-defaults csr > csr.json

生成配置文件模板

cat > ca-config.json <<EOF
    {
      "signing": {
        "default": {
          "expiry": "87600h"
        },
        "profiles": {
          "kubernetes": {
            "usages": [
                "signing",
                "key encipherment",
                "server auth",
                "client auth"
            ],
            "expiry": "87600h"
          }
        }
      }
    }
    EOF

expiry： 其中expiry是过期时间，可以指定多个;  
signing： 表示该证书可以签名其他证书;  
server auth： 表示客户端可以用该证书对服务端进行验证;  
client auth： 表示服务端可以用该证书对客户端进行验证;

--------------------

#### 创建CA证书签名请求 ####

创建ca-csr.json文件：

{
      "CN": "kubernetes",
      "key": { "algo": "rsa", "size": 2048 },
      "names": [ { "C": "CN", "ST": "ZheJiang", "L": "HangZhou", "O": "k8s", "OU": "System" } ] }

*  “CN”：Common Name，kube-apiserver 从证书中提取该字段作为请求的用户名 (User Name)；浏览器使用该字段验证网站是否合法；
 *  “O”：Organization，kube-apiserver 从证书中提取该字段作为请求用户所属的组 (Group)；

--------------------

#### 生成 CA 证书和私钥 ####

[root@wecloud-test-k8s-1 ssl]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca
    2018/04/01 17:16:31 [INFO] generating a new CA key and certificate from CSR
    2018/04/01 17:16:31 [INFO] generate received request
    2018/04/01 17:16:31 [INFO] received CSR
    2018/04/01 17:16:31 [INFO] generating key: rsa-2048
    2018/04/01 17:16:31 [INFO] encoded CSR
    2018/04/01 17:16:31 [INFO] signed certificate with serial number 249255656321361266442489392967889104825701924957
    [root@wecloud-test-k8s-1 ssl]# ls
    ca-config.json  ca.csr  ca-csr.json  ca-key.pem  ca.pem  config.json  csr.json

ca-key.pem 和 ca.pem分别为生成的私钥和证书。

--------------------

### 创建kubernetes证书 ###

创建 kubernetes 证书签名请求文件 kubernetes-csr.json：

{
        "CN": "kubernetes",
        "hosts": [ "127.0.0.1", "192.168.99.183", "192.168.99.189", "192.168.99.185", "192.168.99.196", "10.254.0.1", "kubernetes", "kubernetes.default", "kubernetes.default.svc", "kubernetes.default.svc.cluster", "kubernetes.default.svc.cluster.local" ],
        "key": { "algo": "rsa", "size": 2048 },
        "names": [ { "C": "CN", "ST": "ZheJiang", "L": "HangZhou", "O": "k8s", "OU": "System" } ] }

hosts 字段不为空则需要指定授权使用该证书的 IP 或域名列表,把我们整个集群需要的节点都添加进去。

--------------------

#### 生成kubernetes证书和私钥 ####

[root@wecloud-test-k8s-1 ssl]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kubernetes-csr.json | cfssljson -bare kubernetes

查看生成结果：

[root@wecloud-test-k8s-1 ssl]# ls
    ca-config.json  ca-csr.json  ca.pem       csr.json        kubernetes-csr.json  kubernetes.pem
    ca.csr          ca-key.pem   config.json  kubernetes.csr  kubernetes-key.pem

--------------------

### 创建 admin 证书 ###

创建 admin 证书签名请求文件 admin-csr.json：

{
      "CN": "admin",
      "hosts": [],
      "key": { "algo": "rsa", "size": 2048 },
      "names": [ { "C": "CN", "ST": "BeiJing", "L": "BeiJing", "O": "system:masters", "OU": "System" } ] }

后续 kube-apiserver 使用 RBAC 对客户端(如 kubelet、kube-proxy、Pod)请求进行授权；  
kube-apiserver 预定义了一些 RBAC 使用的 RoleBindings，如 cluster-admin 将 Group system:masters 与 Role cluster-admin 绑定，该 Role 授予了调用kube-apiserver 的所有 API的权限；  
O 指定该证书的 Group 为 system:masters，kubelet 使用该证书访问 kube-apiserver 时 ，由于证书被 CA 签名，所以认证通过，同时由于证书用户组为经过预授权的 system:masters，所以被授予访问所有 API 的权限；

#### 生成 admin 证书和私钥 ####

[root@wecloud-test-k8s-1 ssl]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin
    2018/04/01 17:52:44 [INFO] generate received request
    2018/04/01 17:52:44 [INFO] received CSR
    2018/04/01 17:52:44 [INFO] generating key: rsa-2048
    2018/04/01 17:52:45 [INFO] encoded CSR
    2018/04/01 17:52:45 [INFO] signed certificate with serial number 304064711980818042733176492155385650115390405387
    2018/04/01 17:52:45 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
    websites. For more information see the Baseline Requirements for the Issuance and Management
    of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
    specifically, section 10.2.3 ("Information Requirements").
    [root@wecloud-test-k8s-1 ssl]# ls
    admin.csr       admin-key.pem  ca-config.json  ca-csr.json  ca.pem       csr.json        kubernetes-csr.json  kubernetes.pem
    admin-csr.json  admin.pem      ca.csr          ca-key.pem   config.json  kubernetes.csr  kubernetes-key.pem

--------------------

### 创建 kube-proxy 证书 ###

创建 kube-proxy 证书签名请求文件 kube-proxy-csr.json：

{
      "CN": "system:kube-proxy",
      "hosts": [],
      "key": { "algo": "rsa", "size": 2048 },
      "names": [ { "C": "CN", "ST": "BeiJing", "L": "BeiJing", "O": "k8s", "OU": "System" } ] }

CN 指定该证书的 User 为 system:kube-proxy；  
kube-apiserver 预定义的 RoleBinding cluster-admin 将User system:kube-proxy 与 Role system:node-proxier 绑定，该 Role 授予了调用 kube-apiserver Proxy 相关 API 的权限；

[root@wecloud-test-k8s-1 ssl]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy 
    2018/04/01 18:04:12 [INFO] generate received request
    2018/04/01 18:04:12 [INFO] received CSR
    2018/04/01 18:04:12 [INFO] generating key: rsa-2048
    2018/04/01 18:04:13 [INFO] encoded CSR
    2018/04/01 18:04:13 [INFO] signed certificate with serial number 110046110319381529194553706381534768799655332669
    2018/04/01 18:04:13 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
    websites. For more information see the Baseline Requirements for the Issuance and Management
    of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
    specifically, section 10.2.3 ("Information Requirements").
    [root@wecloud-test-k8s-1 ssl]# ls
    admin.csr       admin.pem       ca-csr.json  config.json     kube-proxy-csr.json  kubernetes.csr       kubernetes.pem
    admin-csr.json  ca-config.json  ca-key.pem   csr.json        kube-proxy-key.pem   kubernetes-csr.json
    admin-key.pem   ca.csr          ca.pem       kube-proxy.csr  kube-proxy.pem       kubernetes-key.pem

--------------------

### 校验证书 ###

以 kubernetes 证书为例

#### 使用 opsnssl 命令 ####

[root@wecloud-test-k8s-1 ssl]# openssl x509 -noout -text -in kubernetes.pem 
    Certificate:
        Data:
            Version: 3 (0x2)
            Serial Number:
                53:0b:be:1f:9d:f1:87:2c:32:29:e7:dd:ec:c7:7b:62:ad:21:24:3b
        Signature Algorithm: sha256WithRSAEncryption
            Issuer: C=CN, ST=ZheJiang, L=HangZhou, O=k8s, OU=System, CN=kubernetes
            Validity
                Not Before: Apr  1 09:25:00 2018 GMT
                Not After : Mar 29 09:25:00 2028 GMT
            Subject: C=CN, ST=ZheJiang, L=HangZhou, O=k8s, OU=System, CN=kubernetes
            Subject Public Key Info:
                Public Key Algorithm: rsaEncryption
                    Public-Key: (2048 bit)
                    Modulus:
                        00:ac:8f:43:bf:e3:9e:a8:1a:4e:43:85:82:1c:ab:
                        d3:33:8a:f3:7f:13:fe:0c:13:ce:6e:95:9b:fb:be:
                        4a:ce:52:75:b0:5d:11:27:3a:b5:6e:83:cb:dd:f9:
                        0b:ee:ee:a9:3b:78:ce:af:03:68:24:8c:76:b3:44:
                        a2:1e:71:6f:3c:3b:97:ff:29:7a:95:7c:25:3f:a2:
                        91:83:8e:75:2f:d1:7e:02:39:23:e8:f4:47:38:d2:
                        6c:9d:d2:80:39:af:d5:b6:cb:cf:da:9b:4f:0e:8e:
                        1f:9e:c0:72:68:47:58:c7:e4:95:3c:74:02:ae:5b:
                        ea:e4:1e:f1:1c:9d:0c:8f:f6:ca:4d:2f:33:bc:31:
                        9a:98:73:0c:b8:c9:9c:3f:5e:5b:94:0c:af:2c:ee:
                        d0:9e:98:e7:dd:d7:af:a4:b0:2e:23:6f:69:2f:8d:
                        fc:06:41:2f:fc:26:37:18:77:16:89:cc:4e:a8:ba:
                        bb:bd:06:4e:48:57:ff:78:3f:f5:2d:e7:05:67:5f:
                        50:d8:ba:ab:40:d5:7b:82:0f:5b:57:bd:00:8d:83:
                        11:da:da:fe:d1:3f:74:f2:74:fc:59:60:e7:e4:47:
                        41:c1:c2:61:c4:35:d2:f3:f4:ba:d9:00:70:a6:37:
                        9c:eb:28:96:e1:34:4c:11:66:9b:9b:6e:86:68:32:
                        6a:c7
                    Exponent: 65537 (0x10001)
            X509v3 extensions:
                X509v3 Key Usage: critical
                    Digital Signature, Key Encipherment
                X509v3 Extended Key Usage: 
                    TLS Web Server Authentication, TLS Web Client Authentication
                X509v3 Basic Constraints: critical
                    CA:FALSE
                X509v3 Subject Key Identifier: 
                    A3:58:12:F2:46:F6:72:15:5D:35:C6:5E:05:5B:23:D1:B6:C4:A2:E1
                X509v3 Authority Key Identifier: 
                    keyid:B9:3B:C0:2D:B4:54:68:AD:69:72:59:51:46:41:54:6B:14:22:FA:33
    
                X509v3 Subject Alternative Name: 
                    DNS:kubernetes, DNS:kubernetes.default, DNS:kubernetes.default.svc, DNS:kubernetes.default.svc.cluster, DNS:kubernetes.default.svc.cluster.local, IP Address:127.0.0.1, IP Address:192.168.99.183, IP Address:192.168.99.189, IP Address:192.168.99.185, IP Address:192.168.99.196, IP Address:10.254.0.1
        Signature Algorithm: sha256WithRSAEncryption
             63:15:06:34:bc:d0:f4:d3:65:f2:01:95:71:8d:d4:d1:6a:6b:
             cc:92:f6:27:dc:34:2d:11:4b:01:78:54:b9:71:df:40:1e:32:
             5d:98:0a:4f:69:a4:0a:8c:42:3a:de:d2:05:6f:90:c6:ce:1b:
             fb:69:28:71:4c:0d:84:64:8e:58:9c:ec:60:fb:52:e7:cb:97:
             dd:45:c6:b3:1a:f2:d0:1b:d6:78:5d:24:10:02:df:ae:72:de:
             6c:c4:cb:e8:8a:f5:50:5c:5c:66:0d:79:4f:95:da:12:5a:ef:
             bf:6c:86:0a:29:ce:c1:f8:62:18:95:2f:03:3d:7d:84:81:5d:
             59:47:1e:6a:78:03:d4:cb:4e:dc:a0:c4:29:54:23:43:a0:1a:
             ab:17:bf:9e:fe:2b:a0:0d:53:49:60:f4:d9:a6:01:b3:83:77:
             17:f8:5d:9a:c7:26:0a:fe:50:51:19:b1:b6:69:eb:d3:7a:aa:
             db:26:a3:fe:2a:b9:89:fb:d1:c5:d5:ad:f2:05:33:83:64:9f:
             fe:5d:fc:2e:34:77:92:4b:2b:fe:86:65:7c:65:14:09:51:f6:
             de:36:b4:93:34:f9:27:60:05:17:c5:c9:4d:e3:65:f1:69:85:
             ec:ef:f9:33:69:70:4b:b8:a0:62:c7:dd:75:89:d6:e6:6d:00:
             4a:7c:1d:8e

*  确认 Issuer 字段的内容和 ca-csr.json 一致；
 *  确认 Subject 字段的内容和 kubernetes-csr.json 一致；
 *  确认 X509v3 Subject Alternative Name 字段的内容和 kubernetes-csr.json 一致；
 *  确认 X509v3 Key Usage、Extended Key Usage 字段的内容和 ca-config.json 中 kubernetes profile 一致；

--------------------

### 分发证书 ###

将生成的证书和密钥文件拷贝到所有机器的/etc/kubernetes/ssl目录下：

[root@wecloud-test-k8s-1 ssl]# mkdir /etc/kubernetes/ssl
    [root@wecloud-test-k8s-1 ssl]# cp *.pem /etc/kubernetes/ssl/

拷贝到另外的几个节点（在此之前需要实现master到其他节点的免密登录）：

[root@wecloud-test-k8s-1 ssl]# scp *.pem 192.168.99.189:/etc/kubernetes/ssl/
    [root@wecloud-test-k8s-1 ssl]# scp *.pem 192.168.99.185:/etc/kubernetes/ssl/
    [root@wecloud-test-k8s-1 ssl]# scp *.pem 192.168.99.196:/etc/kubernetes/ssl/

--------------------

## 小结 ##