AIX系统下RAC故障之一--CRS服务启动失败（coredump）

╰+哭是因爲堅強的太久メ 2022-08-05 11:27 310阅读 0赞

**AIX系统下RAC故障之一--CRS服务启动失败（coredump）**

系统环境：

操作系统： AIX 5300-08

集群软件： CRS 10.2.0.1

数据库：   Oracle 10.2.0.1

**故障现象：**

**在安装完CRS后，运行root.sh脚本出现故障，CRS服务启动失败：**

**\[root@aix224@ install\]\#/u01/crs\_1/root.sh**  
chown: /u01/crs\_1/crs/auth: No such file or directory  
chmod: /u01/crs\_1/crs/auth: No such file or directory  
WARNING: directory '/u01' is not owned by root  
Checking to see if Oracle CRS stack is already configured  
Setting the permissions on OCR backup directory  
Setting up NS directories  
Oracle Cluster Registry configuration upgraded successfully  
chown: /u01/crs\_1/crs/auth: No such file or directory  
chmod: /u01/crs\_1/crs/auth: No such file or directory  
WARNING: directory '/u01' is not owned by root  
clscfg: EXISTING configuration version 3 detected.  
clscfg: version 3 is 10G Release 2.  
Successfully accumulated necessary OCR keys.  
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.  
node <nodenumber>: <nodename> <private interconnect name> <hostname>  
node 1: aix223 aix223-priv aix223  
node 2: aix224 aix224-priv aix224  
clscfg: Arguments check out successfully.  
  
  
NO KEYS WERE WRITTEN. Supply -force parameter to override.  
\-force is destructive and will destroy any previous cluster  
configuration.  
Oracle Cluster Registry for cluster has already been initialized  
Startup will be queued to init within 30 seconds.  
Adding daemons to inittab  
Adding daemons to inittab  
Expecting the CRS daemons to be up within 600 seconds.  
CSS is active on these nodes.  
        aix223  
        aix224  
CSS is active on all nodes.  
Waiting for the Oracle CRSD and EVMD to start  
/u01/crs\_1/install/rootconfig\[965\]: 401480 Abort(coredump)  
Waiting for the Oracle CRSD and EVMD to start  
/u01/crs\_1/install/rootconfig\[965\]: 401486 Abort(coredump)  
Waiting for the Oracle CRSD and EVMD to start  
/u01/crs\_1/install/rootconfig\[965\]: 401492 Abort(coredump)  
Waiting for the Oracle CRSD and EVMD to start  
/u01/crs\_1/install/rootconfig\[965\]: 401498 Abort(coredump)  
Waiting for the Oracle CRSD and EVMD to start  
/u01/crs\_1/install/rootconfig\[965\]: 401504 Abort(coredump)  
Waiting for the Oracle CRSD and EVMD to start  
/u01/crs\_1/install/rootconfig\[965\]: 401510 Abort(coredump)  
Waiting for the Oracle CRSD and EVMD to start  
/u01/crs\_1/install/rootconfig\[965\]: 401516 Abort(coredump)  
Waiting for the Oracle CRSD and EVMD to start  
/u01/crs\_1/install/rootconfig\[965\]: 401522 Abort(coredump)  
Waiting for the Oracle CRSD and EVMD to start  
/u01/crs\_1/install/rootconfig\[965\]: 401528 Abort(coredump)  
Waiting for the Oracle CRSD and EVMD to start  
/u01/crs\_1/install/rootconfig\[965\]: 401534 Abort(coredump)  
Waiting for the Oracle CRSD and EVMD to start  
Waiting for the Oracle CRSD and EVMD to start  
Waiting for the Oracle CRSD and EVMD to start  
Waiting for the Oracle CRSD and EVMD to start  
Waiting for the Oracle CRSD and EVMD to start  
Waiting for the Oracle CRSD and EVMD to start  
Waiting for the Oracle CRSD and EVMD to start  
Waiting for the Oracle CRSD and EVMD to start  
Waiting for the Oracle CRSD and EVMD to start  
Waiting for the Oracle CRSD and EVMD to start  
Waiting for the Oracle CRSD and EVMD to start  
/u01/crs\_1/install/rootconfig\[965\]: 401600 Abort(coredump)  
Waiting for the Oracle CRSD and EVMD to start  
/u01/crs\_1/install/rootconfig\[965\]: 401606 Abort(coredump)  
Waiting for the Oracle CRSD and EVMD to start  
/u01/crs\_1/install/rootconfig\[965\]: 401612 Abort(coredump)  
Waiting for the Oracle CRSD and EVMD to start  
Timed out waiting for the CRS stack to start.

**查看日志信息：**  
\[root@aix224@ install\]\#cd /u01/crs\_1/log/aix224/  
\[root@aix224@ aix224\]\#ls  
admin            client           cssd             racg  
alertaix224.log  crsd             evmd  
**\[root@aix224@ aix224\]\#cd crsd**  
\[root@aix224@ crsd\]\#ls  
**core                      core.2015-09-15-18:20:53  core.2015-09-15-18:22:52  
core.2015-09-15-18:20:05  core.2015-09-15-18:20:59  core.2015-09-15-18:22:58  
core.2015-09-15-18:20:31  core.2015-09-15-18:21:05  core.2015-09-15-18:23:04  
core.2015-09-15-18:20:36  core.2015-09-15-18:21:10  crsd.log  
core.2015-09-15-18:20:42  core.2015-09-15-18:21:16  
core.2015-09-15-18:20:48  core.2015-09-15-18:22:47  
  
解决方法：**

**重新清除OCR配置信息:**

**\[root@aix224@ /\]\# dd if=/dev/zero  of=/dev/rrac\_ocr  bs=8192  count=2560**

**建立以下目录：**  
\[**root@aix224@ /\]\#mkdir -p /u01/crs\_1/crs/auth**   
**\[root@aix224@ /\]\#chgrp dba /u01/crs\_1/crs/auth  
\[root@aix224@ /\]\#chmod 755 /u01/crs\_1/crs/auth  
  
在两个节点重新运行root.sh（aix223、aix224）：**

**\[root@aix224@ install\]\#./rootdelete.sh**  
Shutting down Oracle Cluster Ready Services (CRS):  
Stopping resources.  
Error while stopping resources. Possible cause: CRSD is down.  
Stopping CSSD.  
Unable to communicate with the CSS daemon.  
Shutdown has begun. The daemons should exit soon.  
Checking to see if Oracle CRS stack is down...  
Oracle CRS stack is not running.  
Oracle CRS stack is down now.  
Removing script for Oracle Cluster Ready services  
Updating ocr file for downgrade  
Cleaning up SCR settings in '/etc/oracle/scls\_scr'

**\[root@aix224@ /\]\#/u01/crs\_1/root.sh**  
WARNING: directory '/u01' is not owned by root  
Checking to see if Oracle CRS stack is already configured  
Setting the permissions on OCR backup directory  
Setting up NS directories  
Oracle Cluster Registry configuration upgraded successfully  
WARNING: directory '/u01' is not owned by root  
clscfg: EXISTING configuration version 3 detected.  
clscfg: version 3 is 10G Release 2.  
Successfully accumulated necessary OCR keys.  
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.  
node <nodenumber>: <nodename> <private interconnect name> <hostname>  
node 1: aix223 aix223-priv aix223  
node 2: aix224 aix224-priv aix224  
clscfg: Arguments check out successfully.  
NO KEYS WERE WRITTEN. Supply -force parameter to override.  
\-force is destructive and will destroy any previous cluster  
configuration.  
Oracle Cluster Registry for cluster has already been initialized  
Startup will be queued to init within 30 seconds.  
Adding daemons to inittab  
Adding daemons to inittab  
Expecting the CRS daemons to be up within 600 seconds.  
CSS is active on these nodes.  
        aix223  
        aix224  
CSS is active on all nodes.  
Waiting for the Oracle CRSD and EVMD to start  
Oracle CRS stack installed and running under init(1M)  
Running vipca(silent) for configuring nodeapps  
The given interface(s), "en0, en1" is not public. Public interfaces should be used to configure virtual IPs.  
  
**---脚本运行成功，然后以root用户执行vipca脚本  
  
查看CRS 服务：**

**\[root@aix224@ bin\]\#crsctl check crs  
CSS appears healthy  
CRS appears healthy  
EVM appears healthy**

**\[root@aix224@ bin\]\#crs\_stat -t**  
Name           Type           Target    State     Host          
\------------------------------------------------------------  
ora.aix223.gsd application    ONLINE    ONLINE    aix223        
ora.aix223.ons application    ONLINE    ONLINE    aix223        
ora.aix223.vip application    ONLINE    ONLINE    aix223        
ora.aix224.gsd application    ONLINE    ONLINE    aix224        
ora.aix224.ons application    ONLINE    ONLINE    aix224        
ora.aix224.vip application    ONLINE    ONLINE    aix224

**---CRS服务启动正常！！**