Linux运维之RHCS集群----ISCSI/SCSI实现存储共享、CLVM/GFS2集群文件系统部署

短命女 2021-11-29 16:54 424阅读 0赞

本博文（上）篇为：

[Linux运维之——RHSC集群：Luci Ricci实现高可用Web服务][Linux_RHSC_Luci Ricci_Web]

### 一 环境准备 ###

三台redht6.5的虚拟机。

*  server04 配置luci集群管理界面，用来配置和管理集群。
 *  server05、server2 集群节点，配置ricci。

server03 添加一块虚拟磁盘，配置共享存储，增添一块共享磁盘

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70][]

### 二 搭建步骤 ###

**2.1server3上安装共享存储所需要的软件（高级yum源）**

yum install scsi-* -y

![20190722145539171.png][]

**2.2 在server04、server05上安装客户端iscsi**

yum install iscsi-* -y

![20190722145812564.png][]

**2.3在server3上配置共享策略，开启服务，查看**

vim /etc/tgt/targets.conf 
    
    <target iqn.2019-07.com.example:server.target1>
        backing-store /dev/vda		#确定共享的磁盘
    </target>

![20190722151749788.png][]

/etc/init.d/tgtd start	#启动服务

![20190722151822425.png][]

tgt-admin -s

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 1][]

**2.4 在server04、server05上查看并激活共享的存储**

iscsiadm -m discovery -t st -p 172.25.60.3	#查看共享存储
    iscsiadm -m node -l			#激活共享存储
    fdisk -l

![20190722152359479.png][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 2][]

**2.5 在server04、server05上下载mysql **

yum install mysql-server -y

![20190722152921559.png][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 3][]

**2.6 格式化共享设备（只在一台设备上操作）**

mkfs.ext4 /dev/sdb

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 4][]

**2.7 在server04、server05上更改mysql目录的权限**

mount /dev/sdb /var/lib/mysql/
    ll -d /var/lib/mysql/
    chown mysql.mysql /var/lib/mysql/
    ll -d /var/lib/mysql/

![20190722153457506.png][]

**2.8检测服务能否正常开启**

/etc/init.d/mysqld start
    /etc/init.d/mysqld stop
    umount /dev/sdb

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 5][]

**2.9将apache的单线程关掉，并且把apache服务组资源停掉**

**![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 6][]**

将Run Exclusive的勾去掉，并且重新submit。

也可以用命令：

clusvcadm -d apache

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 7][]

**2.10 浏览器配置（先ip后挂载再httpd服务）**

**2.10.1**在Failover Domains中添加dbfail

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 8][]

2.10.2在Resources中添加IP Address,File System,Script资源

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 9][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 10][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 11][]

2.10.3在Service Groups中添加sql服务组

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 12][]

并添加2.10.2中的三个资源

2.10.4 开启sql服务

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 13][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 14][]

**2.11测试**

**![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 15][]**

clusvcadm -e apache		#启动apache服务组
    clusvcadm -r apache -m server05	#将apache服务组切到server05
    clusvcadm -d apache		#关闭apache服务组

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 16][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 17][]

### 三、CLVM/GFS2集群文件系统部署 ###

！！首先将上个实验的apache、sql服务组的资源停掉

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 18][]

**3.1** **设置CLVM服务开机自启动**

lvmconf --enable-cluster
    cat /etc/lvm/lvm.conf | grep locking_type		#为3则自启动

![20190722195303933.png][]

**3.2** **划分物理卷、逻辑卷组、逻辑卷**

pvcreate /dev/sdb
    vgcreate clustervg /dev/sdb
    lvcreate -L 4G -n demo clustervg
    lvs

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 19][]

**3.3格式化逻辑卷为GFS2集群文件系统**

mkfs.gfs2 -t HA:mygfs2 -p lock_dlm -j 2 /dev/clustervg/demo

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 20][]

mkfs.gfs2  
     -j \#: 指定日志区域的个数，有几个就能够被几个节点所挂载；  
     -J \#: 指定日志区域的大小，默认为128MB;  
     -p \{lock\_dlm|lock\_nolock\}：所使用的锁协议名称，集群需要使用lock\_dlm；  
     -t <name>: 锁表的名称  
                格式为clustername:fsname,  
                clustername为当前节点所在的集群的名称  
                fsname文件系统名称，自定义，要在当前集群惟一，这也叫锁表名称，表明是分布式锁的范围。

**3.4挂载设备，更改目录的权限**

mount /dev/clustervg/demo /var/lib/mysql/
    ll -d /var/lib/mysql/
    chown mysql.mysql /var/lib/mysql/
    blkid

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 21][]

**3.5测试多点挂载**

**server04：**

**![20190722200417862.png][]**

**server05：**

**![20190722200436951.png][]**

**3.6 将组文件永久挂载在/var/lib/mysql**

vim /etc/fstab
    
    UUID="ShOcWz-nCcR-d5i1-mApl-gvV5-3N6T-FIKfjh"     /var/lib/mysql  gfs2    _netdev 0 0		#UUID可以通过blkid查看
    mount -a

**3.7更改浏览器设置**

更改之前先卸载server04/05逻辑卷：
    	umount /dev/clustervg/demo

3.7.1先在Service Groups—sql中删除 Filesystem：

删除后，一定要Submit。

3.7.2删除集群资源中的dbdata

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 22][]

3.7.3重新添加集群资源

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 23][]

3.7.4在Service Groups——sql中添加更改后的gfs2的mygfs：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 24][]

**3.8测试**

启动sql服务组

实现文件共享

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 25][]

[Linux_RHSC_Luci Ricci_Web]: https://blog.csdn.net/vanvan_/article/details/96763278
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70]: /images/20211129/1c0c48aedfc7428ca24a6e9333538191.png
[20190722145539171.png]: /images/20211129/5ba46092ce55491386f3c20f5eb51345.png
[20190722145812564.png]: /images/20211129/9d6b536c67a64bc99de0dd667254edc2.png
[20190722151749788.png]: /images/20211129/a3ba172b33a2455084d836c715948502.png
[20190722151822425.png]: /images/20211129/fa2d5fdc7fce4b02a50a5b91c3576fa5.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 1]: /images/20211129/641993d325564422bb4e072d365a475a.png
[20190722152359479.png]: /images/20211129/f8ecf4f902b5423cbf3a419991db78f2.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 2]: /images/20211129/9baf7d5e1eb14d1b9a2c04e79cec2cb4.png
[20190722152921559.png]: /images/20211129/bb429a310b974bd5b3b17ef965fe86d3.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 3]: /images/20211129/926ba73fcb734b9c956a96d50ee03e81.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 4]: /images/20211129/fa38cc74922744daae94b0f9e812a329.png
[20190722153457506.png]: /images/20211129/ca7901864d6349ab8adc2d8feea2c6a7.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 5]: /images/20211129/15a408d57d664f4b870c73972471602c.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 6]: /images/20211129/d24ed38ea99f45b2865bb7be52871cf7.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 7]: /images/20211129/c1f609095d024cd0ad2ae0961e777357.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 8]: /images/20211129/e8b4f1bd817449e4aa9d5404edbdcadc.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 9]: /images/20211129/61d6e5b4733c46ca851cce4764a7269c.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 10]: /images/20211129/b6faab1df77243ebb2739851319170ae.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 11]: /images/20211129/0c8227110acf4ed284518590db5c6ec4.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 12]: /images/20211129/e45d18a3069f4b8c953ddd4e69eeb6bd.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 13]: /images/20211129/e671890ca6fe490aa06eb47f6f2ad35d.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 14]: /images/20211129/1c2731e066b848139d7638d5abb8f12e.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 15]: /images/20211129/154254a61621418398581d492ca42834.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 16]: /images/20211129/fb559351536a4eef8a4adad855fc01d9.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 17]: /images/20211129/1e8e869dcbbf419bb1fdfd06acc4fedc.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 18]: /images/20211129/99d05fb6456442bd873e49e53eac0c07.png
[20190722195303933.png]: /images/20211129/2a7e5edcc27d4812aa71f55273e1af99.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 19]: /images/20211129/bf6ff17a752746b0b56289f487e2694e.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 20]: /images/20211129/1d70f8e599794493b7adcc22370e27f5.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 21]: /images/20211129/17b319841205437db3e5fb337842b564.png
[20190722200417862.png]: /images/20211129/f6fa3449a86e493e9bbf2955ebe31166.png
[20190722200436951.png]: /images/20211129/eb208b7c8a9b4b3bb03fc0e917c12db5.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 22]: /images/20211129/9864e779d469457992cc5cc2574194bf.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 23]: /images/20211129/af2bdd8d03e24d09ba65e7b48d30151c.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 24]: /images/20211129/e719023a1eeb4831b0a0f65a82fda16e.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3ZhbnZhbl8_size_16_color_FFFFFF_t_70 25]: /images/20211129/879d2cd3c4ec4b959a3bd834cb5250c0.png