使用Kolla镜像Openstack多节点快速搭建部署

缺乏、安全感 2022-06-04 03:38 333阅读 0赞

本文所需要的镜像为陈沙克老师公司制作好的一个镜像文件，直接下载来使用

云盘地址：[https://pan.baidu.com/share/init?surl=kVkilGr][https_pan.baidu.com_share_init_surl_kVkilGr] 密码: gcty

--------------------

在这篇文章([http://blog.csdn.net/dylloveyou/article/details/77148560][http_blog.csdn.net_dylloveyou_article_details_77148560])中，我们已经部署了单节点的OpenStack环境，这次我们尝试部署一个多节点的环境(包括1个controller节点，1个compute节点，1个storage节点)，之前的单节点因为已经安装了kolla-ansible和docker registry，我们把它当做部署节点。

--------------------

## 1. 环境准备 ##

新创建3台虚拟机，分别作为controller节点，compute节点，storage节点。其中controller节点3张网卡，compute、storage节点2张网卡。操作系统为centos7.4

**1）关闭Selinux**

vi /etc/sysconfig/selinux SELINUX=disabled

**2）关闭firewalld**

systemctl stop firewalld
    systemctl disable firewalld

**3）设置主机名，hosts文件**

vi /etc/hosts
    
    192.168.128.77 control01 
    192.168.128.85 controller 
    192.168.128.86 compute 
    192.168.128.87 storage

**4）storage节点加盘**  
storage节点新加一块磁盘，作为cinder的lvm后端

![这里写图片描述][20171203163630816]

加完后重启系统，可以查看到新加的磁盘

[root@storage ~]# fdisk -l
    
    Disk /dev/sdb: 10.7 GB, 10737418240 bytes, 20971520 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    
    Disk /dev/sda: 10.7 GB, 10737418240 bytes, 20971520 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk label type: dos
    Disk identifier: 0x000d78d5
    
       Device Boot      Start         End      Blocks   Id  System
    /dev/sda1            2048    20971519    10484736   8e  Linux LVM
    ......

创建pv、vg

[root@storage ~]# pvcreate /dev/sdb
      Physical volume "/dev/sdb" successfully created.
    [root@storage ~]# vgcreate cinder-volumes /dev/sdb
      Volume group "cinder-volumes" successfully created

修改配置  
vi /etc/lvm/lvm.conf  
修改 devices 下面的

filter = [ "a/sda/", "a/sdb/", "r/.*/"]

重启lvm服务

systemctl restart lvm2-lvmetad.service

**5）确认节点配置**

controller 3 张网卡  
compute 2 张网卡  
storage 2 张网卡(其实一张即可，因为不需要tunnel网络)  
storage 2 块硬盘

![这里写图片描述][20171203163650467]

每个主机的eth0网卡作为 API&Management 网络，使用的是“NAT模式”  
每个主机的eth1网卡作为 Tenant(VM) 网络，是承载VxLAN的底层网络，使用的是“仅主机模式 ”  
controller节点也作为网络节点，需要模拟OpenStack的外网，增加网卡eth2，这里也选择“仅主机模式”

关于如何应用OpenStack的OVS+vxlan网络模式，请参考：[http://blog.csdn.net/dylloveyou/article/details/72639187][http_blog.csdn.net_dylloveyou_article_details_72639187]

关于如何修改CentOS7的网卡名称，请参考：[http://blog.csdn.net/dylloveyou/article/details/78697896][http_blog.csdn.net_dylloveyou_article_details_78697896]

--------------------

## 2.安装docker ##

**1）加入Docker的repo源**

# tee /etc/yum.repos.d/docker.repo << 'EOF'
    [dockerrepo]
    name=Docker Repository
    baseurl=https://yum.dockerproject.org/repo/main/centos/$releasever/
    enabled=1
    gpgcheck=1
    gpgkey=https://yum.dockerproject.org/gpg
    EOF

**2）安装Docker 1.12.6**

yum install docker-engine-1.12.6 docker-engine-selinux-1.12.6 -y

**3）设置Docker**

mkdir /etc/systemd/system/docker.service.d
    tee /etc/systemd/system/docker.service.d/kolla.conf << 'EOF'
    [Service]
    MountFlags=shared
    EOF

**4）重启相关服务**

systemctl daemon-reload
    systemctl enable docker
    systemctl restart docker

**5）编辑/usr/lib/systemd/system/docker.service文件**

ExecStart=/usr/bin/dockerd --insecure-registry 192.168.128.77:4000

**6）重启Docker服务**

systemctl daemon-reload
    systemctl restart docker

--------------------

## 3.配置免密登录 ##

部署节点和其他节点免密登陆

[root@control01 ~]# ssh-keygen 
    [root@control01 ~]# ssh-copy-id controller 
    [root@control01 ~]# ssh-copy-id compute 
    [root@control01 ~]# ssh-copy-id storage

--------------------

## 4.部署OpenStack ##

**1）停止部署节点容器**

因为部署节点之前已经部署了All in One的OpenStack，为了降低虚拟机的资源占用(我是用笔记本上面的虚拟机做的实验)，先停止本节的的OpenStack相关容器。

[root@control01 ~]# cd /root/kolla-ansible-4.0.3.dev36/tools/
    [root@control01 tools]# . kolla-ansible stop

查看只剩registry容器在运行

[root@control01 tools]# docker ps
    CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
    b18c5accabc2        registry            "/bin/registry serve "   3 months ago        Up 22 minutes       0.0.0.0:4000->5000/tcp   registry

**2）修改multinode文件**

[root@control01 ~]# cd /root/kolla-ansible-4.0.3.dev36/
    [root@control01 kolla-ansible-4.0.3.dev36]# cp ansible/inventory/* /home/
    [root@control01 ~]# cd /home
    [root@control01 home]# ls
    all-in-one  multinode
    [root@control01 home]# cp multinode mymultinode

[root@control01 home]# vi mymultinode
    
    [control]
    controller  
    [network]
    controller
    [compute]
    compute
    [monitoring]
    controller
    [storage]
    storage
    ......

里面的内容很好理解，表示一个控制节点 controller，网络节点也安装到控制节点，一个计算节点 compute ，一个存储节点 storage，后面的部分不用修改

**3）修改global和password配置文件**

[root@control01 ~]# cd /etc/kolla/
    [root@control01 kolla]# vi globals.yml

kolla_internal_vip_address: "192.168.128.85" 外部访问地址，如果是非HA环境，是控制节点IP
    
    docker_registry: "192.168.128.77:4000" 镜像库，用部署节点的镜像库 docker_namespace: "99cloud" network_interface: "eth0" 网络接口，默认就这一个接口，这里我们以eth0作为API网络 api_interface: "{ { network_interface }}" 用的是eth0 tunnel_interface: "eth1" eth1作为vxlan的承载网络 neutron_external_interface: "eth2" eth2作为外部网络 enable_cinder: "yes" 启用cinder enable_cinder_backend_lvm: "yes" cinder后端用lvm cinder_volume_group: "cinder-volumes" enable_haproxy: "no" 不启用haproxy

[root@control01 kolla]# vi passwords.yml
    keystone_admin_password: admin  admin用户登录密码

**4）部署前检查**

[root@control01 ~]# cd /root/kolla-ansible-4.0.3.dev36/tools/
    [root@control01 ~]# . kolla-ansible prechecks -i /home/mymultinode

报错 ImportError: No module named docker

TASK [prechecks : Checking docker-py version] **********************************
    fatal: [compute]: FAILED! => {
       "changed": false, "cmd": ["/usr/bin/python", "-c", "import docker; print docker.__version__"], "delta": "0:00:00.012605", "end": "2017-12-02 20:57:39.611343", "failed": true, "failed_when_result": true, "rc": 1, "start": "2017-12-02 20:57:39.598738", "stderr": "Traceback (most recent call last):\n File \"<string>\", line 1, in <module>\nImportError: No module named docker", "stdout": "", "stdout_lines": [], "warnings": []}
    ......

解决：在各个节点安装 python-docker-py

yum install python-docker-py

再次执行检查，没有问题，开始部署

**5）部署**

[root@control01 tools]# . kolla-ansible deploy -i /home/mymultinode

![这里写图片描述][20171203163719162]

开始部署，等待大约20分钟，部署完成，一般不会有什么问题，如果有问题，则需要根据报错及容器日志查找原因

登录dashboard  
[http://192.168.128.85][http_192.168.128.85]

![这里写图片描述][SouthEast]

登录系统，查看各个服务正常，表明部署成功。

可以通过docker ps命令查看各个节点运行的容器

[root@controller ~]# docker ps
    CONTAINER ID        IMAGE                                                                         COMMAND             CREATED             STATUS              PORTS               NAMES
    a344f712eb4f        192.168.128.77:4000/99cloud/centos-source-horizon:4.0.2.1                     "kolla_start"       43 minutes ago      Up 24 minutes                           horizon
    7089aea91586        192.168.128.77:4000/99cloud/centos-source-heat-engine:4.0.2.1                 "kolla_start"       45 minutes ago      Up 45 minutes                           heat_engine
    071f259e23dc        192.168.128.77:4000/99cloud/centos-source-heat-api-cfn:4.0.2.1                "kolla_start"       45 minutes ago      Up 45 minutes                           heat_api_cfn
    d75bb72de87a        192.168.128.77:4000/99cloud/centos-source-heat-api:4.0.2.1                    "kolla_start"       45 minutes ago      Up 45 minutes                           heat_api
    824215d900c8        192.168.128.77:4000/99cloud/centos-source-neutron-metadata-agent:4.0.2.1      "kolla_start"       47 minutes ago      Up 47 minutes                           neutron_metadata_agent
    c5f9262efdc5        192.168.128.77:4000/99cloud/centos-source-neutron-l3-agent:4.0.2.1            "kolla_start"       47 minutes ago      Up 47 minutes                           neutron_l3_agent
    b2b47375d344        192.168.128.77:4000/99cloud/centos-source-neutron-dhcp-agent:4.0.2.1          "kolla_start"       47 minutes ago      Up 47 minutes                           neutron_dhcp_agent
    9a36954ca062        192.168.128.77:4000/99cloud/centos-source-neutron-openvswitch-agent:4.0.2.1   "kolla_start"       47 minutes ago      Up 47 minutes                           neutron_openvswitch_agent
    0b8bc000ccb3        192.168.128.77:4000/99cloud/centos-source-neutron-server:4.0.2.1              "kolla_start"       47 minutes ago      Up 47 minutes                           neutron_server
    f822f81226d1        192.168.128.77:4000/99cloud/centos-source-openvswitch-vswitchd:4.0.2.1        "kolla_start"       47 minutes ago      Up 47 minutes                           openvswitch_vswitchd
    63af3bdb8e2d        192.168.128.77:4000/99cloud/centos-source-openvswitch-db-server:4.0.2.1       "kolla_start"       48 minutes ago      Up 48 minutes                           openvswitch_db
    b0b1a42a1a17        192.168.128.77:4000/99cloud/centos-source-nova-novncproxy:4.0.2.1             "kolla_start"       50 minutes ago      Up 50 minutes                           nova_novncproxy
    97277a9a1d14        192.168.128.77:4000/99cloud/centos-source-nova-consoleauth:4.0.2.1            "kolla_start"       50 minutes ago      Up 50 minutes                           nova_consoleauth
    1be7b07ba70a        192.168.128.77:4000/99cloud/centos-source-nova-conductor:4.0.2.1              "kolla_start"       50 minutes ago      Up 50 minutes                           nova_conductor
    ae94b6f70ebc        192.168.128.77:4000/99cloud/centos-source-nova-scheduler:4.0.2.1              "kolla_start"       50 minutes ago      Up 50 minutes                           nova_scheduler
    5975962a3c33        192.168.128.77:4000/99cloud/centos-source-nova-api:4.0.2.1                    "kolla_start"       50 minutes ago      Up 50 minutes                           nova_api
    0de1f8e33774        192.168.128.77:4000/99cloud/centos-source-nova-placement-api:4.0.2.1          "kolla_start"       50 minutes ago      Up 50 minutes                           placement_api
    0efb76129caa        192.168.128.77:4000/99cloud/centos-source-cinder-scheduler:4.0.2.1            "kolla_start"       57 minutes ago      Up 57 minutes                           cinder_scheduler
    2d50185586a8        192.168.128.77:4000/99cloud/centos-source-cinder-api:4.0.2.1                  "kolla_start"       57 minutes ago      Up 57 minutes                           cinder_api
    5c4e87b1db04        192.168.128.77:4000/99cloud/centos-source-glance-registry:4.0.2.1             "kolla_start"       58 minutes ago      Up 58 minutes                           glance_registry
    635bc1f305a9        192.168.128.77:4000/99cloud/centos-source-glance-api:4.0.2.1                  "kolla_start"       58 minutes ago      Up 58 minutes                           glance_api
    de6ec58f6d6e        192.168.128.77:4000/99cloud/centos-source-keystone:4.0.2.1                    "kolla_start"       About an hour ago   Up About an hour                        keystone
    f881fe857541        192.168.128.77:4000/99cloud/centos-source-rabbitmq:4.0.2.1                    "kolla_start"       About an hour ago   Up About an hour                        rabbitmq
    ef6941e59581        192.168.128.77:4000/99cloud/centos-source-mariadb:4.0.2.1                     "kolla_start"       About an hour ago   Up About an hour                        mariadb
    022b6935fc16        192.168.128.77:4000/99cloud/centos-source-cron:4.0.2.1                        "kolla_start"       About an hour ago   Up About an hour                        cron
    acdc83df5d52        192.168.128.77:4000/99cloud/centos-source-kolla-toolbox:4.0.2.1               "kolla_start"       About an hour ago   Up About an hour                        kolla_toolbox
    6f0a2b4fc44e        192.168.128.77:4000/99cloud/centos-source-fluentd:4.0.2.1                     "kolla_start"       About an hour ago   Up About an hour                        fluentd
    2b2f15895c0a        192.168.128.77:4000/99cloud/centos-source-memcached:4.0.2.1                   "kolla_start"       About an hour ago   Up About an hour                        memcached

[root@compute ~]# docker ps
    CONTAINER ID        IMAGE                                                                         COMMAND             CREATED             STATUS              PORTS               NAMES
    e99bdf6a6513        192.168.128.77:4000/99cloud/centos-source-neutron-openvswitch-agent:4.0.2.1   "kolla_start"       48 minutes ago      Up 48 minutes                           neutron_openvswitch_agent
    90621685744e        192.168.128.77:4000/99cloud/centos-source-openvswitch-vswitchd:4.0.2.1        "kolla_start"       48 minutes ago      Up 48 minutes                           openvswitch_vswitchd
    537a8f2dbf06        192.168.128.77:4000/99cloud/centos-source-openvswitch-db-server:4.0.2.1       "kolla_start"       49 minutes ago      Up 49 minutes                           openvswitch_db
    63761ee44b3c        192.168.128.77:4000/99cloud/centos-source-nova-compute:4.0.2.1                "kolla_start"       51 minutes ago      Up 51 minutes                           nova_compute
    53b1784996c7        192.168.128.77:4000/99cloud/centos-source-nova-libvirt:4.0.2.1                "kolla_start"       51 minutes ago      Up 51 minutes                           nova_libvirt
    292c7bcbd170        192.168.128.77:4000/99cloud/centos-source-nova-ssh:4.0.2.1                    "kolla_start"       52 minutes ago      Up 52 minutes                           nova_ssh
    64d2031a345d        192.168.128.77:4000/99cloud/centos-source-iscsid:4.0.2.1                      "kolla_start"       About an hour ago   Up About an hour                        iscsid
    04b2c9bd71e6        192.168.128.77:4000/99cloud/centos-source-cron:4.0.2.1                        "kolla_start"       About an hour ago   Up About an hour                        cron
    c22eede98b72        192.168.128.77:4000/99cloud/centos-source-kolla-toolbox:4.0.2.1               "kolla_start"       About an hour ago   Up About an hour                        kolla_toolbox
    2478ceaa712a        192.168.128.77:4000/99cloud/centos-source-fluentd:4.0.2.1                     "kolla_start"       About an hour ago   Up About an hour                        fluentd

[root@storage ~]# docker ps
    CONTAINER ID        IMAGE                                                             COMMAND             CREATED             STATUS              PORTS               NAMES
    91607ec544b1        192.168.128.77:4000/99cloud/centos-source-cinder-backup:4.0.2.1   "kolla_start"       55 minutes ago      Up 55 minutes                           cinder_backup
    b40ad28f6f8b        192.168.128.77:4000/99cloud/centos-source-cinder-volume:4.0.2.1   "kolla_start"       55 minutes ago      Up 55 minutes                           cinder_volume
    61ca356cbab1        192.168.128.77:4000/99cloud/centos-source-tgtd:4.0.2.1            "kolla_start"       About an hour ago   Up About an hour                        tgtd
    40e86a921d22        192.168.128.77:4000/99cloud/centos-source-iscsid:4.0.2.1          "kolla_start"       About an hour ago   Up About an hour                        iscsid
    0133da5c2ba9        192.168.128.77:4000/99cloud/centos-source-cron:4.0.2.1            "kolla_start"       About an hour ago   Up About an hour                        cron
    3bd6802e44ed        192.168.128.77:4000/99cloud/centos-source-kolla-toolbox:4.0.2.1   "kolla_start"       About an hour ago   Up About an hour                        kolla_toolbox
    e0187c061936        192.168.128.77:4000/99cloud/centos-source-fluentd:4.0.2.1         "kolla_start"       About an hour ago   Up Abo

--------------------

本文转载于：[http://blog.csdn.net/dylloveyou/article/details/78699235][http_blog.csdn.net_dylloveyou_article_details_78699235]

[https_pan.baidu.com_share_init_surl_kVkilGr]: https://pan.baidu.com/share/init?surl=kVkilGr
[http_blog.csdn.net_dylloveyou_article_details_77148560]: http://blog.csdn.net/dylloveyou/article/details/77148560
[20171203163630816]: /images/20220604/4bfa1e34422e423fbed70b6a9dfaac29.png
[20171203163650467]: /images/20220604/2cc7587612fd4b29acae137323d2e1a2.png
[http_blog.csdn.net_dylloveyou_article_details_72639187]: http://blog.csdn.net/dylloveyou/article/details/72639187
[http_blog.csdn.net_dylloveyou_article_details_78697896]: http://blog.csdn.net/dylloveyou/article/details/78697896
[20171203163719162]: /images/20220604/eb68ee6366aa40ad97b7be273cb3fd8c.png
[http_192.168.128.85]: http://192.168.128.85
[SouthEast]: /images/20220604/a27e6c89beae4720bbf37cda0e83d27d.png
[http_blog.csdn.net_dylloveyou_article_details_78699235]: http://blog.csdn.net/dylloveyou/article/details/78699235