使用 kubeadm 在 GCP 部署 Kubernetes

约定不等于承诺〃 2022-02-14 16:21 448阅读 0赞

Kubernetes

0. 介绍

最近在准备 CKA 考试，所以需要搭建一个 Kubernetes 集群来方便练习．GCP 平台新用户注册送 300 刀体验金，所以就想到用 kubeadm 在 GCP 弄个练练手，既方便又省钱．

这一套做下来，还是比较容易上手的，kubeadm 提供的是傻瓜式的安装体验，所以难度主要还是在科学上网和熟悉 GCP 的命令上，接下来就详细记述一下如何操作．

1. 准备

接下来的操作都假设已经设置好了科学上网，由于政策原因，具体做法请自行搜索；而且已经注册好了 GCP 账户，链接如下：GCP

1.1 gcloud 安装和配置

首先需要在本地电脑上安装 GCP 命令行客户端：gcloud，参考链接为：gcloud

因为众所周知的原因，gcloud 要能正常使用，要设置代理才可以，下面是设置 SOCKS5 代理的命令：

  1. # gcloud config set proxy/type PROXY_TYPE
  2. $ gcloud config set proxy/type socks5
  3. # gcloud config set proxy/address PROXY_IP_ADDRESS
  4. $ gcloud config set proxy/address 127.0.0.1
  5. # gcloud config set proxy/port PROXY_PORT
  6. $ gcloud config set proxy/address 1080

如果是第一次使用 GCP，需要先进行初始化．在初始化的过程中会有几次交互，使用默认选项即可．由于之前已经设置了代理，网络代理相关部分就可以跳过了．
注意：在选择 region（区域）时，建议选择 us-west2，原因是目前大部分 GCP 的 region，体验用户只能最多创建四个虚拟机实例，只有少数几个区域可以创建六个，其中就包括 us-west2，正常来讲，搭建 Kubernetes 需要三个 master，三个 worker，四个不太够用，当然如果只是试试的话，两个节点，一主一从，也够用了．

  1. $ gcloud init 
  2. Welcome! This command will take you through the configuration of gcloud.
  4. Settings from your current configuration [profile-name] are:
  5. core:
  6.   disable_usage_reporting: 'True'
  8. Pick configuration to use:
  9.  [1] Re-initialize this configuration [profile-name] with new settings 
  10.  [2] Create a new configuration
  11.  [3] Switch to and re-initialize existing configuration: [default]
  12. Please enter your numeric choice:  3
  14. Your current configuration has been set to: [default]
  16. You can skip diagnostics next time by using the following flag:
  17.   gcloud init --skip-diagnostics
  19. Network diagnostic detects and fixes local network connection issues.
  20. Checking network connection...done.                                                                                                                                                                               
  21. ERROR: Reachability Check failed.
  22.     Cannot reach https://www.google.com (ServerNotFoundError)
  23.     Cannot reach https://accounts.google.com (ServerNotFoundError)
  24.     Cannot reach https://cloudresourcemanager.googleapis.com/v1beta1/projects (ServerNotFoundError)
  25.     Cannot reach https://www.googleapis.com/auth/cloud-platform (ServerNotFoundError)
  26.     Cannot reach https://dl.google.com/dl/cloudsdk/channels/rapid/components-2.json (ServerNotFoundError)
  27. Network connection problems may be due to proxy or firewall settings.
  29. Current effective Cloud SDK network proxy settings:
  30.     type = socks5
  31.     host = PROXY_IP_ADDRESS
  32.     port = 1080
  33.     username = None
  34.     password = None
  36. What would you like to do?
  37.  [1] Change Cloud SDK network proxy properties
  38.  [2] Clear all gcloud proxy properties
  39.  [3] Exit
  40. Please enter your numeric choice:  1
  42. Select the proxy type:
  43.  [1] HTTP
  44.  [2] HTTP_NO_TUNNEL
  45.  [3] SOCKS4
  46.  [4] SOCKS5
  47. Please enter your numeric choice:  4
  49. Enter the proxy host address: 127.0.0.1
  51. Enter the proxy port: 1080
  53. Is your proxy authenticated (y/N)?  N
  55. Cloud SDK proxy properties set.
  57. Rechecking network connection...done.                                                                                                                                                                             
  58. Reachability Check now passes.
  59. Network diagnostic (1/1 checks) passed.
  61. You must log in to continue. Would you like to log in (Y/n)?  y
  63. Your browser has been opened to visit:
  65.     https://accounts.google.com/o/oauth2/auth?redirect_uri=......
  68. 已在现有的浏览器会话中创建新的窗口。
  71. Updates are available for some Cloud SDK components.  To install them,
  72. please run:
  73.   $ gcloud components update
  75. You are logged in as: [<gmail account>].
  77. Pick cloud project to use: 
  78.  [1] <project-id>
  79.  [2] Create a new project
  80. Please enter numeric choice or text value (must exactly match list 
  81. item):  1
  83. Your current project has been set to: [<project-id>].
  85. Your project default Compute Engine zone has been set to [us-west2-b].
  86. You can change it by running [gcloud config set compute/zone NAME].
  88. Your project default Compute Engine region has been set to [us-west2].
  89. You can change it by running [gcloud config set compute/region NAME].
  91. Created a default .boto configuration file at [/home/<username>/.boto]. See this file and
  92. [https://cloud.google.com/storage/docs/gsutil/commands/config] for more
  93. information about configuring Google Cloud Storage.
  94. Your Google Cloud SDK is configured and ready to use!
  96. * Commands that require authentication will use <gmail account> by default
  97. * Commands will reference project `<project-id>` by default
  98. * Compute Engine commands will use region `us-west2` by default
  99. * Compute Engine commands will use zone `us-west2-b` by default
  101. Run `gcloud help config` to learn how to change individual settings
  103. This gcloud configuration is called [default]. You can create additional configurations if you work with multiple accounts and/or projects.
  104. Run `gcloud topic configurations` to learn more.
  106. Some things to try next:
  108. * Run `gcloud --help` to see the Cloud Platform services you can interact with. And run `gcloud help COMMAND` to get help on any gcloud command.
  109. * Run `gcloud topic -h` to learn about advanced features of the SDK like arg files and output formatting

1.2 GCP 资源创建

接下来创建 Kuernetes 所需的 GCP 资源．

第一步是创建网络和子网．

  1. $ gcloud compute networks create cka --subnet-mode custom
  2. $ gcloud compute networks subnets create kubernetes --network cka --range 10.240.0.0/24

接下来要创建防火墙规则，配置哪些端口是可以开放访问的．一共两条规则，一个外网，一个内网．
外网规则只需要开放 ssh, ping 和 kube-api 的访问就足够了：

  1. $ gcloud compute firewall-rules create cka-external --allow tcp:22,tcp:6443,icmp --network cka --source-ranges 0.0.0.0/0

内网规则设置好 GCP 虚拟机网段和后面 pod 的网段可以互相访问即可，因为后面会使用 calico 作为网络插件，所以只开放 TCP, UDP 和 ICMP 是不够的，还需要开放 BGP，但 GCP 的防火墙规则中没有 BGP 选项，所以放开全部协议的互通．

  1. $ gcloud compute firewall-rules create cka-internal --network cka --allow=all --source-ranges 192.168.0.0/16,10.240.0.0/16

最后创建 GCP 虚拟机实例．

  1. $ gcloud compute instances create controller-1 --async --boot-disk-size 200GB --can-ip-forward --image-family ubuntu-1804-lts --image-project ubuntu-os-cloud --machine-type n1-standard-1 --private-network-ip 10.240.0.11 --scopes compute-rw,storage-ro,service-management,service-control,logging-write,monitoring --subnet kubernetes --tags cka,controller
  2. $ gcloud compute instances create worker-1 --async --boot-disk-size 200GB --can-ip-forward --image-family ubuntu-1804-lts --image-project ubuntu-os-cloud --machine-type n1-standard-1 --private-network-ip 10.240.0.21 --scopes compute-rw,storage-ro,service-management,service-control,logging-write,monitoring --subnet kubernetes --tags cka,worker

2. 主节点配置

使用 gcloud 登录 controller-1

  1. $ gcloud compute ssh controller-1
  2. WARNING: The public SSH key file for gcloud does not exist.
  3. WARNING: The private SSH key file for gcloud does not exist.
  4. WARNING: You do not have an SSH key for gcloud.
  5. WARNING: SSH keygen will be executed to generate a key.
  6. Generating public/private rsa key pair.
  7. Enter passphrase (empty for no passphrase): 
  8. Enter same passphrase again: 
  9. Your identification has been saved in /home/<username>/.ssh/google_compute_engine.
  10. Your public key has been saved in /home/<username>/.ssh/google_compute_engine.pub.
  11. The key fingerprint is:
  12. SHA256:jpaZtzz42t7FjB1JV06GeVHhXVi12LF/a+lfl7TK2pw <username>@<username>
  13. The key's randomart image is:
  14. +---[RSA 2048]----+
  15. |               O&|
  16. |              B=B|
  17. |            ...*o|
  18. |           . o  .|
  19. |        S   o  .o|
  20. |       *   = .. *|
  21. |      *.o . =  *o|
  22. |     ..+.o .+ = o|
  23. |      .+*....E .o|
  24. +----[SHA256]-----+
  25. Updating project ssh metadata...⠧Updated [https://www.googleapis.com/compute/v1/projects/<project-id>].                                                                                                  
  26. Updating project ssh metadata...done.                                                                                                                                                                             
  27. Waiting for SSH key to propagate.
  28. Warning: Permanently added 'compute.2329485573714771968' (ECDSA) to the list of known hosts.
  29. Welcome to Ubuntu 18.04.1 LTS (GNU/Linux 4.15.0-1025-gcp x86_64)
  31.  * Documentation:  https://help.ubuntu.com
  32.  * Management:     https://landscape.canonical.com
  33.  * Support:        https://ubuntu.com/advantage
  35.   System information as of Wed Dec  5 03:05:31 UTC 2018
  37.   System load:  0.0               Processes:           87
  38.   Usage of /:   1.2% of 96.75GB   Users logged in:     0
  39.   Memory usage: 5%                IP address for ens4: 10.240.0.11
  40.   Swap usage:   0%
  43.   Get cloud support with Ubuntu Advantage Cloud Guest:
  44.     http://www.ubuntu.com/business/services/cloud
  46. 0 packages can be updated.
  47. 0 updates are security updates.
  50. $ ssh -l<user-name> -i .ssh/google_compute_engine.pub 35.236.126.174

安装 kubeadm, docker, kubelet, kubectl．

  1. $ sudo apt update
  2. $ sudo apt upgrade -y
  3. $ sudo apt-get install -y docker.io
  5. $ sudo vim /etc/apt/sources.list.d/kubernetes.list
  6. deb http://apt.kubernetes.io/ kubernetes-xenial main
  8. $ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo  apt-key add -
  9. OK
  11. $ sudo apt update
  12. $ sudo apt-get install -y \
  13. kubeadm=1.12.2-00 kubelet=1.12.2-00 kubectl=1.12.2-00

kubeadm 初始化

  1. $ sudo kubeadm init --pod-network-cidr 192.168.0.0/16

配置 calico 网络插件

  1. $ wget https://tinyurl.com/yb4xturm \
  2. -O rbac-kdd.yaml
  3. $ wget https://tinyurl.com/y8lvqc9g \
  4. -O calico.yaml
  5. $ kubectl apply -f rbac-kdd.yaml
  6. $ kubectl apply -f calico.yaml

配置 kubectl 的 bash 自动补全．

  1. $ source <(kubectl completion bash)
  2. $ echo "source <(kubectl completion bash)" >> ~/.bashrc

3. 从节点配置

这里偷懒了一下，从节点安装的包和主节点一模一样，大家可以根据需求，去掉一些不必要的包．

  1. $ sudo apt-get update && sudo apt-get upgrade -y
  2. $ apt-get install -y docker.io
  4. $ sudo vim /etc/apt/sources.list.d/kubernetes.list
  5. deb http://apt.kubernetes.io/ kubernetes-xenial main
  7. $ curl -s \
  8. https://packages.cloud.google.com/apt/doc/apt-key.gpg \
  9. | sudo apt-key add -
  10. $ sudo apt-get update
  11. $ sudo apt-get install -y \
  12. kubeadm=1.12.2-00 kubelet=1.12.2-00 kubectl=1.12.2-00

如果此时 kubeadm init 命令中的 join 命令找不到了，或者 bootstrap token 过期了，该怎么办呢，下面就是解决方法．

  1. $ sudo kubeadm token list
  2. TOKEN TTL EXPIRES USAGES DESCRIPTION
  3. 27eee4.6e66ff60318da929 23h 2017-11-03T13:27:33Z
  4. authentication,signing The default bootstrap token generated
  5. by ’kubeadm init’....
  7. $ sudo kubeadm token create
  8. 27eee4.6e66ff60318da929
  10. $ openssl x509 -pubkey \
  11. -in /etc/kubernetes/pki/ca.crt | openssl rsa \
  12. -pubin -outform der 2>/dev/null | openssl dgst \
  13. -sha256 -hex | sed ’s/^.* //’
  14. 6d541678b05652e1fa5d43908e75e67376e994c3483d6683f2a18673e5d2a1b0

最后执行 kubeadm join 命令．

  1. $ sudo kubeadm join \
  2. --token 27eee4.6e66ff60318da929 \
  3. 10.128.0.3:6443 \
  4. --discovery-token-ca-cert-hash \
  5. sha256:6d541678b05652e1fa5d43908e75e67376e994c3483d6683f2a18673e5d2a1b0

4. 参考文档

  • GCP Cloud SDK 安装指南
  • 配置 Cloud SDK 以在代理/防火墙后使用
  • Kubernetes the hard way
  • Linux Academy: Certified Kubernetes Administrator (CKA)

发表评论

表情：
评论列表 （有 0 条评论，448人围观）

还没有评论，来说两句吧...

相关阅读