K8s单Master部署

素颜马尾好姑娘i 2023-10-04 08:41 43阅读 0赞

文章目录

  • 一、配置etcd
    • 1.1 制作etcd证书文件
      • 1.1.1 下载cfssl
      • 1.1.2 生成etcd证书文件
    • 1.2 配置etcd
    • 1.3 node1，node2上配置etcd
  • 二、flannel网络配置

K8s单Master部署

  • 环境：一台master，ip:20.0.0.10 安装etcd

  • 两台node节点：

    • node1：ip：20.0.0.11 安装etcd,flannel
    • node2：ip：20.0.0.12 安装etcd,flannel

##部署etcd思路：需要下载etcd二进制文件，解压使用其中的etcd和etcdctl可执行文件；需要etcd证书文件，使用cfssl证书工具；需要编辑etcd配置文件，搭建etcd群集。
##部署flannel网络思路：将要配置的节点ip写入etcd中，以便flannel能够识别。构建开启flannel网络并修改docker service文件使得docker能够支持flannel网络。

一、配置etcd

1.1 制作etcd证书文件

1.1.1 下载cfssl

  1. curl -L https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -o /usr/local/bin/cfssl
  2. curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /usr/local/bin/cfssljson
  3. curl -L https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 -o /usr/local/bin/cfssl-certinfo
  4. chmod +x /usr/local/bin/cfssl /usr/local/bin/cfssljson /usr/local/bin/cfssl-certinfo
  6. 文件放在/usr/local/bin/目录下方便执行

1.1.2 生成etcd证书文件

编辑shell脚本来制作证书文件

  1. mkdir /root/k8s/etcd-cert/
  2. cd /root/k8s/etcd-cert/
  4. vim etcd-cert.sh
  5. cat > ca-config.json <<EOF
  6. {
  7.   "signing": {
  8.     "default": {
  9.       "expiry": "87600h"
  10.     },
  11.     "profiles": {
  12.       "www": {
  13.          "expiry": "87600h",
  14.          "usages": [
  15.             "signing",
  16.             "key encipherment",
  17.             "server auth",
  18.             "client auth"
  19.         ]
  20.       }
  21.     }
  22.   }
  23. }
  24. EOF
  26. cat > ca-csr.json <<EOF
  27. {
  28.     "CN": "etcd CA",
  29.     "key": {
  30.         "algo": "rsa",
  31.         "size": 2048
  32.     },
  33.     "names": [
  34.         {
  35.             "C": "CN",
  36.             "L": "Nanjing",
  37.             "ST": "Nanjing"
  38.         }
  39.     ]
  40. }
  41. EOF
  43. cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
  45. #-----------------------
  47. cat > server-csr.json <<EOF
  48.     "hosts": [
  49.     "20.0.0.10",
  50.     "20.0.0.11",
  51.     "20.0.0.12"
  52.     ],
  53.     "key": {
  54.         "algo": "rsa",
  55.         "size": 2048
  56.     },
  57.     "names": [
  58.         {
  59.             "C": "CN",
  60.             "L": "Nanjing",
  61.             "ST": "Nanjing"
  62.         }
  63.     ]
  64. }
  65. EOF
  67. cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server

执行脚本生成证书

  1. sh /root/k8s/etcd-cert/etcd-cert.sh

1.2 配置etcd

  1. 下载etcd二进制文件
  2. etcd下载地址 https://github.com/etcd-io/etcd/releases 
  3.  tar zxvf etcd-v3.3.10-linux-amd64.tar.gz
  4. mkdir -p /opt/etcd/{bin,cfg,ssl}
  5. cp k8s/etcd-cert/*.pem /opt/etcd/ssl/
  6.  mv etcd etcdctl /opt/etcd/bin/
  8.  编写配置etcd的shell脚本
  9.  vim etcd.sh
  10.  #!/bin/bash
  12. ETCD_NAME=$1
  13. ETCD_IP=$2
  14. ETCD_CLUSTER=$3
  16. WORK_DIR=/opt/etcd
  18. cat <<EOF >$WORK_DIR/cfg/etcd
  19. #[Member]
  20. ETCD_NAME="${ETCD_NAME}"
  21. ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
  22. ETCD_LISTEN_PEER_URLS="https://${ETCD_IP}:2380"
  23. ETCD_LISTEN_CLIENT_URLS="https://${ETCD_IP}:2379"
  25. #[Clustering]
  26. ETCD_INITIAL_ADVERTISE_PEER_URLS="https://${ETCD_IP}:2380"
  27. ETCD_ADVERTISE_CLIENT_URLS="https://${ETCD_IP}:2379"
  28. ETCD_INITIAL_CLUSTER="etcd01=https://${ETCD_IP}:2380,${ETCD_CLUSTER}"
  29. ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
  30. ETCD_INITIAL_CLUSTER_STATE="new"
  31. EOF
  33. cat <<EOF >/usr/lib/systemd/system/etcd.service
  34. [Unit]
  35. Description=Etcd Server
  36. After=network.target
  37. After=network-online.target
  38. Wants=network-online.target
  40. [Service]
  41. Type=notify
  42. EnvironmentFile=${WORK_DIR}/cfg/etcd
  43. ExecStart=${WORK_DIR}/bin/etcd \
  44. --name=\${ETCD_NAME} \
  45. --data-dir=\${ETCD_DATA_DIR} \
  46. --listen-peer-urls=\${ETCD_LISTEN_PEER_URLS} \
  47. --listen-client-urls=\${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 \
  48. --advertise-client-urls=\${ETCD_ADVERTISE_CLIENT_URLS} \
  49. --initial-advertise-peer-urls=\${ETCD_INITIAL_ADVERTISE_PEER_URLS} \
  50. --initial-cluster=\${ETCD_INITIAL_CLUSTER} \
  51. --initial-cluster-token=\${ETCD_INITIAL_CLUSTER_TOKEN} \
  52. --initial-cluster-state=new \
  53. --cert-file=${WORK_DIR}/ssl/server.pem \
  54. --key-file=${WORK_DIR}/ssl/server-key.pem \
  55. --peer-cert-file=${WORK_DIR}/ssl/server.pem \
  56. --peer-key-file=${WORK_DIR}/ssl/server-key.pem \
  57. --trusted-ca-file=${WORK_DIR}/ssl/ca.pem \
  58. --peer-trusted-ca-file=${WORK_DIR}/ssl/ca.pem
  59. Restart=on-failure
  60. LimitNOFILE=65536
  62. [Install]
  63. WantedBy=multi-user.target
  64. EOF
  66. ##启动etcd
  67. systemctl daemon-reload
  68. systemctl enable etcd
  69. systemctl restart etcd

执行脚本

  1. sh etcd.sh etcd01 20.0.0.10 etcd02=https://20.0.0.11:2380,etcd03=https://20.0.0.12:2380
  3. netstat -anpt | grep etcd
  4. 此时只显示只配置了一个etcd节点，还没有配置etcd集群。
  6. master上的etcd配置完毕

1.3 node1，node2上配置etcd

  1. ##配置其他两个node节点上的etcd，只需要将master上的/opt/etcd/目录和/usr/lib/systemd/system/etcd.service文件scp到各节点上。
  2. 然后修改配置文件/opt/etcd/cfg/etcd中的节点name和节点的ip即可。然后启动etcd服务即可。
  4. master节点上操作
  5. scp -r /opt/etcd/ root@20.0.0.11:/opt/
  6. scp -r /opt/etcd/ root@20.0.0.12:/opt/
  7. scp /usr/lib/systemd/system/etcd.service root@20.0.0.11:/usr/lib/systemd/system/
  8. scp /usr/lib/systemd/system/etcd.service root@20.0.0.12:/usr/lib/systemd/system/
  10. ##去到node1，node2节点上更改配置
  12. node1配置如下，node2同理
  13. root@node1 ~]# tree /opt/etcd/
  14. /opt/etcd/
  15. ├── bin
  16. │   ├── etcd
  17. │   └── etcdctl
  18. ├── cfg
  19. │   └── etcd
  20. └── ssl
  21.     ├── ca-key.pem
  22.     ├── ca.pem
  23.     ├── server-key.pem
  24.     └── server.pem
  26. 3 directories, 7 files
  28. vim /opt/etcd/cfg/etcd 
  29. #[Member]
  30. ETCD_NAME="etcd02"
  31. ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
  32. ETCD_LISTEN_PEER_URLS="https://20.0.0.11:2380"
  33. ETCD_LISTEN_CLIENT_URLS="https://20.0.0.11:2379"
  35. #[Clustering]
  36. ETCD_INITIAL_ADVERTISE_PEER_URLS="https://20.0.0.11:2380"
  37. ETCD_ADVERTISE_CLIENT_URLS="https://20.0.0.11:2379"
  38. ETCD_INITIAL_CLUSTER="etcd01=https://20.0.0.10:2380,etcd02=https://20.0.0.11:2380,etcd03=https://20.0.0.12:2380"
  39. ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
  40. ETCD_INITIAL_CLUSTER_STATE="new"
  42. ##启动etcd
  43. systemctl start etcd
  44. systemctl enable etcd
  46. 然后在master上检查集群健康状态
  47.  /opt/etcd/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://20.0.0.10:2379,https://20.0.0.11:2379,https://20.0.0.12:2379" cluster-health
  48.  检查通过，etcd二进制集群配置完成。

二、flannel网络配置

  1. ##在任意配置了etcd的节点上写入分配的子网到etcd中，供flannel使用
  3. cd /opt/etcd/ssl/
  4. /opt/etcd/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://20.0.0.10:2379,https://20.0.0.11:2379,https://20.0.0.12:2379" set /coreos.com/network/config '{"Network": "172.17.0.0/16","Backend": {"Type": "vxlan"}}'
  6. ##查看
  7. /opt/etcd/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://20.0.0.10:2379,https://20.0.0.11:2379,https://20.0.0.12:2379" get /coreos.com/network/config
  9. ##node1，node2上导入flannel二进制文件包，
  10. tar zxvf flannel-v0.10.0-linux-amd64.tar.gz -C /tmp/
  11. mkdir /opt/kubernetes/{cfg,bin,ssl} -p
  12. cd /tmp
  13. mv mk-docker-opts.sh flanneld /opt/kubernetes/bin/
  15. ##编写搭建flannel网络脚本
  17. vim flannel.sh
  19. #!/bin/bash
  21. ETCD_ENDPOINTS=${1:-"http://127.0.0.1:2379"}
  23. cat <<EOF >/opt/kubernetes/cfg/flanneld
  25. FLANNEL_OPTIONS="--etcd-endpoints=${ETCD_ENDPOINTS} \
  26. -etcd-cafile=/opt/etcd/ssl/ca.pem \
  27. -etcd-certfile=/opt/etcd/ssl/server.pem \
  28. -etcd-keyfile=/opt/etcd/ssl/server-key.pem"
  30. EOF
  32. cat <<EOF >/usr/lib/systemd/system/flanneld.service
  33. [Unit]
  34. Description=Flanneld overlay address etcd agent
  35. After=network-online.target network.target
  36. Before=docker.service
  38. [Service]
  39. Type=notify
  40. EnvironmentFile=/opt/kubernetes/cfg/flanneld
  41. ExecStart=/opt/kubernetes/bin/flanneld --ip-masq \$FLANNEL_OPTIONS
  42. ExecStartPost=/opt/kubernetes/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/subnet.env
  43. Restart=on-failure
  45. [Install]
  46. WantedBy=multi-user.target
  48. EOF
  50. systemctl daemon-reload
  51. systemctl enable flanneld
  52. systemctl restart flanneld

##开启flannel网络功能

  1. sh flannel.sh https://20.0.0.10:2379,https://20.0.0.11:2379,https://20.0.0.12:2379

##配置docker连接flannel网络

  1. vim /usr/lib/systemd/system/docker.service
  3. ...
  4. # for containers run by docker
  5. ####添加如下两条语句
  6. EnvironmentFile=/run/flannel/subnet.env
  7. Execstart=/usr/bin/dockerd $DOCKER_NETWORK_OPTIONS -H fd:// --containerd=/run/containerd/containerd.sock
  8. ...

注：同时可以再/run/flannel/subnet.env文件下看到flannel子网环境变量
##重启docker服务

  1. systemctl daemon-reload
  2. systemctl restart docker
  4. ##查看flannel网络
  5. ifconfig

##验证：两节点上分别创建容器，进入容器中查看网络，同时两容器互相ping测试是否能ping通

  1. docker run -it centos:7 /bin/bash
  2. yum -y install net-tools 
  3. ifconfig    ##容器中查看网络
  5. ping另一个节点的容器ip，如果能ping通则表示flannel网络构建成功，docker可以识别flannel网络。

发表评论

表情：
评论列表 （有 0 条评论，43人围观）

还没有评论，来说两句吧...

相关阅读