oracle vm上安装oracle11g r2 rac（11.2.0.4）

左手的ㄟ右手 2022-06-04 06:24 308阅读 0赞

oracle 11g R2 rac安装（11.2.0.4）  
  
环境：  
2个节点rac部署  
hostname:rac01    
操作系统：CentOS release 6.5 (Final)    
内存：1837496 kB  
磁盘：30G  
hostname:rac02    
操作系统：CentOS release 6.5 (Final)    
内存：1837496 kB  
磁盘：3G  
数据库软件：  
p13390677\_112040\_Linux-x86-64\_1of7  
p13390677\_112040\_Linux-x86-64\_2of7  
p13390677\_112040\_Linux-x86-64\_3of7  
  
安装虚拟机安装操作系统(省略)  
  
1.主机环境检查  
内存  
\[root@rac01 ~\]\# grep MemTotal /proc/meminfo  
MemTotal:        1837496 kB  
swap  
\[root@rac01 ~\]\# grep SwapTotal /proc/meminfo  
SwapTotal:       3071992 kB  
oracle要求swap大小：  
Between 2.5 GB and 32 GB Equal to the size of RAM  
More than 32 GB           32 GB of RAM  
磁盘  
\[root@rac01 ~\]\# df -h  
Filesystem      Size  Used Avail Use% Mounted on  
/dev/sda2        20G  2.9G   16G  16% /  
tmpfs           898M  300K  897M   1% /dev/shm  
/dev/sda1       194M   30M  155M  16% /boot  
/dev/sda3       5.8G  140M  5.4G   3% /home  
/dev/sda6       2.0G   35M  1.8G   2% /tmp  
\[root@rac01 ~\]\# df -h /tmp  
Filesystem      Size  Used Avail Use% Mounted on  
/dev/sda6       2.0G   35M  1.8G   2% /tmp  
2.关闭防火墙  
\[root@rac01 ~\]\# service iptables status  
Chain INPUT (policy ACCEPT)  
num  target     prot opt source               destination           
1    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED   
2    ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0             
3    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0             
4    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:22   
5    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited   
Chain FORWARD (policy ACCEPT)  
num  target     prot opt source               destination           
1    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited   
Chain OUTPUT (policy ACCEPT)  
num  target     prot opt source               destination           
\[root@rac01 ~\]\#  service iptables stop   
\[root@rac01 ~\]\# chkconfig iptables off  
\[root@rac01 ~\]\#  service iptables status  
3.关闭selinux  
查看selinux状态  
\[root@rac01 ~\]\#  /usr/sbin/sestatus -v  
SELinux status:                 enabled  
SELinuxfs mount:                /selinux  
Current mode:                   enforcing  
Mode from config file:          enforcing  
Policy version:                 24  
Policy from config file:        targeted  
Process contexts:  
Current context:                unconfined\_u:unconfined\_r:unconfined\_t:s0-s0:c0.c1023  
Init context:                   system\_u:system\_r:init\_t:s0  
/sbin/mingetty                  system\_u:system\_r:getty\_t:s0  
/usr/sbin/sshd                  system\_u:system\_r:sshd\_t:s0-s0:c0.c1023  
File contexts:  
Controlling term:               unconfined\_u:object\_r:user\_devpts\_t:s0  
/etc/passwd                     system\_u:object\_r:etc\_t:s0  
/etc/shadow                     system\_u:object\_r:shadow\_t:s0  
/bin/bash                       system\_u:object\_r:shell\_exec\_t:s0  
/bin/login                      system\_u:object\_r:login\_exec\_t:s0  
/bin/sh                         system\_u:object\_r:bin\_t:s0 -> system\_u:object\_r:shell\_exec\_t:s0  
/sbin/agetty                    system\_u:object\_r:getty\_exec\_t:s0  
/sbin/init                      system\_u:object\_r:init\_exec\_t:s0  
/sbin/mingetty                  system\_u:object\_r:getty\_exec\_t:s0  
/usr/sbin/sshd                  system\_u:object\_r:sshd\_exec\_t:s0  
查看selinux的模式  
\[root@rac01 ~\]\# getenforce  
Enforcing  
\[root@rac01 ~\]\# cat /etc/selinux/config  
\# This file controls the state of SELinux on the system.  
\# SELINUX= can take one of these three values:  
\#     enforcing - SELinux security policy is enforced.  
\#     permissive - SELinux prints warnings instead of enforcing.  
\#     disabled - No SELinux policy is loaded.  
SELINUX=enforcing  
\# SELINUXTYPE= can take one of these two values:  
\#     targeted - Targeted processes are protected,  
\#     mls - Multi Level Security protection.  
SELINUXTYPE=targeted   
关闭selinux  
\[root@rac01 ~\]\# sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config  
\[root@rac01 ~\]\# cat /etc/selinux/config  
\# This file controls the state of SELinux on the system.  
\# SELINUX= can take one of these three values:  
\#     enforcing - SELinux security policy is enforced.  
\#     permissive - SELinux prints warnings instead of enforcing.  
\#     disabled - No SELinux policy is loaded.  
SELINUX=disabled  
\# SELINUXTYPE= can take one of these two values:  
\#     targeted - Targeted processes are protected,  
\#     mls - Multi Level Security protection.  
SELINUXTYPE=targeted   
临时性关闭  
\[root@rac01 ~\]\# setenforce 0  
  
  
4.网络ip规划：  
127.0.0.1       localhost localhost.localdomain localhost4 localhost4.localdomain4  
\#::1         localhost localhost.localdomain localhost6 localhost6.localdomain6  
10.38.1.182     rac01  
10.38.1.85      rac02  
\#private ip  
192.168.56.102  rac01-priv  
192.168.56.101  rac02-priv  
\#public ip  
10.38.1.56      rac01-vip  
10.38.1.59      rac02-vip  
\#scan ip  
10.38.1.188     scanip  
  
5.创建用户（oracle，grid）创建用户组（oinstall,asmadmin,asmdba,asmoper）以及设置用户环境变量：  
\#!/bin/bash  
\#Purpose:Create 6 groups named 'oinstall','dba','asmadmin','asmdba','asmoper','oper', plus 2 users named 'oracle','grid'.  
\#Also setting the Environment  
\#variable for oracle user.  
\#variable for grid user.  
\#Usage:Log on as the superuser('root'),and then execute the command:\#./1preusers.sh  
\#Author:Asher Huang  
echo "Now create 6 groups named 'oinstall','dba','asmadmin','asmdba','asmoper','oper'"  
echo "Plus 2 users named 'oracle','grid',Also setting the Environment"  
groupadd -g 1000 oinstall  
groupadd -g 1200 asmadmin  
groupadd -g 1201 asmdba  
groupadd -g 1202 asmoper  
useradd -u 1100 -g oinstall -G asmadmin,asmdba,asmoper -d /home/grid -s /bin/bash -c "grid Infrastructure Owner" grid  
echo "grid" | passwd --stdin grid  
echo 'export PS1="\`/bin/hostname -s\`-> "'>> /home/grid/.bash\_profile  
echo "export TMP=/tmp">> /home/grid/.bash\_profile  
echo 'export TMPDIR=$TMP'>>/home/grid/.bash\_profile  
echo "export ORACLE\_SID=+ASM1">> /home/grid/.bash\_profile  
echo "export ORACLE\_BASE=/u01/app/grid">> /home/grid/.bash\_profile  
echo "export ORACLE\_HOME=/u01/app/11.2.0/grid">> /home/grid/.bash\_profile  
echo "export NLS\_DATE\_FORMAT='yyyy/mm/dd hh24:mi:ss'" >> /home/grid/.bash\_profile  
echo 'export TNS\_ADMIN=$ORACLE\_HOME/network/admin' >> /home/grid/.bash\_profile  
echo 'export PATH=/usr/sbin:$PATH'>> /home/grid/.bash\_profile  
echo 'export PATH=$ORACLE\_HOME/bin:$PATH'>> /home/grid/.bash\_profile  
echo 'export LD\_LIBRARY\_PATH=$ORACLE\_HOME/lib:/lib:/usr/lib'>> /home/grid/.bash\_profile  
echo 'export CLASSPATH=$ORACLE\_HOME/JRE:$ORACLE\_HOME/jlib:$ORACLE\_HOME/rdbms/jlib'>> /home/grid/.bash\_profile  
echo "export EDITOR=vi" >> /home/grid/.bash\_profile  
echo "export LANG=en\_US" >> /home/grid/.bash\_profile  
echo "export NLS\_LANG=american\_america.AL32UTF8" >> /home/grid/.bash\_profile  
echo "umask 022">> /home/grid/.bash\_profile  
groupadd -g 1300 dba  
groupadd -g 1301 oper  
useradd -u 1101 -g oinstall -G dba,oper,asmdba -d /home/oracle -s /bin/bash -c "Oracle Software Owner" oracle  
echo "oracle" | passwd --stdin oracle  
echo 'export PS1="\`/bin/hostname -s\`-> "'>> /home/oracle/.bash\_profile  
echo "export TMP=/tmp">> /home/oracle/.bash\_profile  
echo 'export TMPDIR=$TMP'>>/home/oracle/.bash\_profile  
echo "export ORACLE\_HOSTNAME=node2">> /home/oracle/.bash\_profile  
echo "export ORACLE\_SID=rac01">> /home/oracle/.bash\_profile  
echo "export ORACLE\_BASE=/u01/app/oracle">> /home/oracle/.bash\_profile  
echo 'export ORACLE\_HOME=$ORACLE\_BASE/product/11.2.0/db\_1'>> /home/oracle/.bash\_profile  
echo 'export TNS\_ADMIN=$ORACLE\_HOME/network/admin' >> /home/oracle/.bash\_profile  
echo 'export PATH=/usr/sbin:$PATH'>> /home/oracle/.bash\_profile  
echo 'export PATH=$ORACLE\_HOME/bin:$PATH'>> /home/oracle/.bash\_profile  
echo 'export LD\_LIBRARY\_PATH=$ORACLE\_HOME/lib:/lib:/usr/lib'>> /home/oracle/.bash\_profile  
echo 'export CLASSPATH=$ORACLE\_HOME/JRE:$ORACLE\_HOME/jlib:$ORACLE\_HOME/rdbms/jlib' >> /home/oracle/.bash\_profile  
echo "export EDITOR=vi" >> /home/oracle/.bash\_profile  
echo "export LANG=en\_US" >> /home/oracle/.bash\_profile  
echo "export NLS\_LANG=american\_america.AL32UTF8" >> /home/oracle/.bash\_profile  
echo "export NLS\_DATE\_FORMAT='yyyy/mm/dd hh24:mi:ss'" >> /home/oracle/.bash\_profile  
echo "umask 022">> /home/oracle/.bash\_profile  
echo "The Groups and users has been created"  
echo "The Environment for grid,oracle also has been set successfully"  
  
6.创建目录grid\_base,grid\_home,oracle\_base  
\#!/bin/bash  
\#Purpose:Create the necessary directory for oracle,grid users and change the authention to oracle,grid users.  
\#Usage:Log on as the superuser('root'),and then execute the command:\#./2predir.sh  
\#Author:Asher Huang  
echo "Now create the necessary directory for oracle,grid users and change the authention to oracle,grid users..."  
mkdir -p /u01/app/grid  
mkdir -p /u01/app/11.2.0/grid  
mkdir -p /u01/app/oracle  
chown -R oracle:oinstall /u01  
chown -R grid:oinstall /u01/app/grid  
chown -R grid:oinstall /u01/app/11.2.0  
chmod -R 775 /u01  
echo "The necessary directory for oracle,grid users and change theauthention to oracle,grid users has been finished"  
  
7.调整资源限制limts.conf  
\#!/bin/bash  
\#Purpose:Change the /etc/security/limits.conf.  
\#Usage:Log on as the superuser('root'),and then execute the command:\#./3prelimits.sh  
\#Author:Asher Huang  
echo "Now modify the /etc/security/limits.conf,but backup it named /etc/security/limits.conf.bak before"  
cp /etc/security/limits.conf /etc/security/limits.conf.bak  
echo "oracle soft nproc 2047" >>/etc/security/limits.conf  
echo "oracle hard nproc 16384" >>/etc/security/limits.conf  
echo "oracle soft nofile 1024" >>/etc/security/limits.conf  
echo "oracle hard nofile 65536" >>/etc/security/limits.conf  
echo "grid soft nproc 2047" >>/etc/security/limits.conf  
echo "grid hard nproc 16384" >>/etc/security/limits.conf  
echo "grid soft nofile 1024" >>/etc/security/limits.conf  
echo "grid hard nofile 65536" >>/etc/security/limits.conf  
echo "Modifing the /etc/security/limits.conf has been succeed."  
  
8.设置/etc/pam.d/login.  
\#!/bin/bash  
\#Purpose:Modify the /etc/pam.d/login.  
\#Usage:Log on as the superuser('root'),and then execute the command:\#./4prelimits.sh  
\#Author:Asher Huang  
echo "Now modify the /etc/pam.d/login,but with a backup named /etc/pam.d/login.bak"  
cp /etc/pam.d/login /etc/pam.d/login.bak  
echo "session required /lib/security/pam\_limits.so" >>/etc/pam.d/login  
echo "session required pam\_limits.so" >>/etc/pam.d/login  
echo "Modifing the /etc/pam.d/login has been succeed."  
  
9.设置/etc/profile  
\#!/bin/bash  
\#Purpose:Modify the /etc/profile.  
\#Usage:Log on as the superuser('root'),and then execute the command:\#./5preprofile.sh  
\#Author:Asher Huang  
echo "Now modify the /etc/profile,but with a backup named /etc/profile.bak"  
cp /etc/profile /etc/profile.bak  
echo 'if \[ $USER = "oracle" \]||\[ $USER = "grid" \]; then' >> /etc/profile  
echo 'if \[ $SHELL = "/bin/ksh" \]; then' >> /etc/profile  
echo 'ulimit -p 16384' >> /etc/profile  
echo 'ulimit -n 65536' >> /etc/profile  
echo 'else' >> /etc/profile  
echo 'ulimit -u 16384 -n 65536' >> /etc/profile  
echo 'fi' >> /etc/profile  
echo 'fi' >> /etc/profile  
echo "Modifing the /etc/profile has been succeed."  
  
10.设置内核参数：  
\#!/bin/bash  
\#Purpose:Modify the /etc/sysctl.conf.  
\#Usage:Log on as the superuser('root'),and then execute the command:\#./6presysctl.sh  
\#Author:Asher Huang  
echo "Now modify the /etc/sysctl.conf,but with a backup named /etc/sysctl.bak"  
cp /etc/sysctl.conf /etc/sysctl.conf.bak  
echo "fs.aio-max-nr = 1048576" >> /etc/sysctl.conf  
echo "fs.file-max = 6815744" >> /etc/sysctl.conf  
echo "kernel.shmall = 2097152" >> /etc/sysctl.conf  
echo "kernel.shmmax = 1054472192" >> /etc/sysctl.conf  
echo "kernel.shmmni = 4096" >> /etc/sysctl.conf  
echo "kernel.sem = 250 32000 100 128" >> /etc/sysctl.conf  
echo "net.ipv4.ip\_local\_port\_range = 9000 65500" >> /etc/sysctl.conf  
echo "net.core.rmem\_default = 262144" >> /etc/sysctl.conf  
echo "net.core.rmem\_max = 4194304" >> /etc/sysctl.conf  
echo "net.core.wmem\_default = 262144" >> /etc/sysctl.conf  
echo "net.core.wmem\_max = 1048586" >> /etc/sysctl.conf  
echo "net.ipv4.tcp\_wmem = 262144 262144 262144" >> /etc/sysctl.conf  
echo "net.ipv4.tcp\_rmem = 4194304 4194304 4194304" >> /etc/sysctl.conf  
echo "Modifing the /etc/sysctl.conf has been succeed."  
echo "Now make the changes take effect....."  
sysctl –p  
  
11.安装操作系统相应包  
Linux x86-64 Oracle Grid Infrastructure and Oracle RAC Package Requirements  
The following packages (or later versions) must be installed:  
binutils-2.20.51.0.2-5.11.el6 (x86\_64)  
compat-libcap1-1.10-1 (x86\_64)  
compat-libstdc++-33-3.2.3-69.el6 (x86\_64)  
compat-libstdc++-33-3.2.3-69.el6.i686  
gcc-4.4.4-13.el6 (x86\_64)  
gcc-c++-4.4.4-13.el6 (x86\_64)  
glibc-2.12-1.7.el6 (i686)  
glibc-2.12-1.7.el6 (x86\_64)  
glibc-devel-2.12-1.7.el6 (x86\_64)  
glibc-devel-2.12-1.7.el6.i686  
ksh  
libgcc-4.4.4-13.el6 (i686)  
libgcc-4.4.4-13.el6 (x86\_64)  
libstdc++-4.4.4-13.el6 (x86\_64)  
libstdc++-4.4.4-13.el6.i686  
libstdc++-devel-4.4.4-13.el6 (x86\_64)  
libstdc++-devel-4.4.4-13.el6.i686  
libaio-0.3.107-10.el6 (x86\_64)  
libaio-0.3.107-10.el6.i686  
libaio-devel-0.3.107-10.el6 (x86\_64)  
libaio-devel-0.3.107-10.el6.i686  
make-3.81-19.el6  
sysstat-9.0.4-11.el6 (x86\_64)  
  
Checking the Software Requirements  
\[root@rac01 home\]\# cat /proc/version  
Linux version 2.6.32-431.el6.x86\_64 (mockbuild@c6b8.bsys.dev.centos.org) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-4) (GCC) ) \#1 SMP Fri Nov 22 03:15:09 UTC 2013  
\[root@rac01 home\]\# uname -r  
2.6.32-431.el6.x86\_64  
\[root@rac01 home\]\# rpm -q binutils compat-libstdc++ elfutils gcc glibc libaio libgcc libstdc++ compat-libcap1-1 gcc-4 gcc-c++  glibc-devel libstdc++-devel libaio-devel make sysstat unixodbc  
binutils-2.20.51.0.2-5.36.el6.x86\_64  
package compat-libstdc++ is not installed  
elfutils-0.152-1.el6.x86\_64  
package gcc is not installed  
glibc-2.12-1.132.el6.x86\_64  
libaio-0.3.107-10.el6.x86\_64  
libgcc-4.4.7-4.el6.x86\_64  
libstdc++-4.4.7-4.el6.x86\_64  
package compat-libcap1-1 is not installed  
package gcc-4 is not installed  
package gcc-c++ is not installed  
glibc-devel-2.12-1.132.el6.x86\_64  
package libstdc++-devel is not installed  
package libaio-devel is not installed  
make-3.81-20.el6.x86\_64  
sysstat-9.0.4-22.el6.x86\_64  
package unixodbc is not installed  
\[root@rac01 home\]\#  
  
12.克隆虚拟机 第二个rac节点  
Oracle VM VirtualBOX下克隆虚拟机镜像  
Windows环境下在Oracle VM VirtualBOX下克隆虚拟机镜像：  
1.通过命令操作  
VBoxManage.exe clonevdi F:\\mysqlvm\\mysqledi.vdi F:\\mysqlvm\\mysqlwms.vdi  
F:\\Program Files\\Oracle\\VirtualBox>VBoxManage.exe clonevdi F:\\mysqlvm\\mysqledi.v  
di F:\\mysqlvm\\mysqlwms.vdi  
VBoxManage.exe: error: Failed to lock source media 'F:\\mysqlvm\\mysqledi.vdi'  
VBoxManage.exe: error: Details: code VBOX\_E\_INVALID\_OBJECT\_STATE (0x80bb0007), c  
omponent MediumWrap, interface IMedium, callee IUnknown  
VBoxManage.exe: error: Context: "CloneTo(pDstMedium, ComSafeArrayAsInParam(l\_var  
iants), NULL, pProgress.asOutParam())" at line 950 of file VBoxManageDisk.cpp  
上述报错是因为被克隆的虚拟机没有停止  
停止虚拟机再次执行克隆命令  
F:\\Program Files\\Oracle\\VirtualBox>VBoxManage.exe clonevdi F:\\mysqlvm\\mysqledi.v  
di F:\\mysqlvm\\mysqlwms.vdi  
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%  
Clone medium created in format 'VDI'. UUID: 4b3399f7-9466-4d42-9314-2654db46c833  
2.在VirtualBOX新建虚拟机（导入克隆.vdi文件）：  
“新建虚拟机”->“虚拟硬盘”->“使用已有的虚拟硬盘文件(U)”,选择：名称  
  
修改新克隆出来的系统的hostname 为rac02  
\[root@rac01 ~\]\#   
\[root@rac01 ~\]\# vi /etc/sysconfig/network  
NETWORKING=yes  
HOSTNAME=rac02  
"/etc/sysconfig/network" 2L, 30C written  
\[root@rac01 ~\]\# hostname rac02  
\[root@rac01 ~\]\#  
修改克隆虚拟机的网卡名字  
\[root@rac02 ~\]\# ifconfig  
eth2      Link encap:Ethernet  HWaddr 08:00:27:E6:71:80    
          inet addr:10.0.2.15  Bcast:10.0.2.255  Mask:255.255.255.0  
          inet6 addr: fe80::a00:27ff:fee6:7180/64 Scope:Link  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1  
          RX packets:9 errors:0 dropped:0 overruns:0 frame:0  
          TX packets:49 errors:0 dropped:0 overruns:0 carrier:0  
          collisions:0 txqueuelen:1000   
          RX bytes:2660 (2.5 KiB)  TX bytes:4644 (4.5 KiB)  
  
eth3      Link encap:Ethernet  HWaddr 08:00:27:5B:E9:D5    
          inet addr:10.38.1.85  Bcast:10.38.1.255  Mask:255.255.255.0  
          inet6 addr: fe80::a00:27ff:fe5b:e9d5/64 Scope:Link  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1  
          RX packets:2242 errors:0 dropped:0 overruns:0 frame:0  
          TX packets:145 errors:0 dropped:0 overruns:0 carrier:0  
          collisions:0 txqueuelen:1000   
          RX bytes:202350 (197.6 KiB)  TX bytes:20123 (19.6 KiB)  
  
  
\[root@rac02 ~\]\# vi /etc/udev/rules.d/70-persistent-net.rules  
\# This file was automatically generated by the /lib/udev/write\_net\_rules  
\# program, run by the persistent-net-generator.rules rules file.  
\#  
\# You can modify it, as long as you keep each rule on a single  
\# line, and change only the value of the NAME= key.  
  
\# PCI device 0x8086:0x100e (e1000)  
\#SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?\*", ATTR\{address\}=="08:00:27:30:97:a1", ATTR\{type\}=="1", KERNEL=="eth\*", NAME="eth0"  
  
\# PCI device 0x8086:0x100e (e1000)  
\#SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?\*", ATTR\{address\}=="08:00:27:e7:9d:4a", ATTR\{type\}=="1", KERNEL=="eth\*", NAME="eth1"  
  
\# PCI device 0x8086:0x100e (e1000)  
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?\*", ATTR\{address\}=="08:00:27:e6:71:80", ATTR\{type\}=="1", KERNEL=="eth\*", NAME="eth1"  
  
\# PCI device 0x8086:0x100e (e1000)  
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?\*", ATTR\{address\}=="08:00:27:5b:e9:d5", ATTR\{type\}=="1", KERNEL=="eth\*", NAME="eth0"  
~  
  
修改完成后，重启系统reboot  
  
\[root@rac02 ~\]\# ifconfig  
eth0      Link encap:Ethernet  HWaddr 08:00:27:5B:E9:D5    
          inet addr:10.38.1.85  Bcast:11.255.255.255  Mask:254.0.0.0  
          inet6 addr: fe80::a00:27ff:fe5b:e9d5/64 Scope:Link  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1  
          RX packets:9631 errors:0 dropped:0 overruns:0 frame:0  
          TX packets:1225 errors:0 dropped:0 overruns:0 carrier:0  
          collisions:0 txqueuelen:1000   
          RX bytes:736935 (719.6 KiB)  TX bytes:86099 (84.0 KiB)  
  
eth1      Link encap:Ethernet  HWaddr 08:00:27:E6:71:80    
          inet addr:10.0.2.16  Bcast:11.255.255.255  Mask:254.0.0.0  
          inet6 addr: fe80::a00:27ff:fee6:7180/64 Scope:Link  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1  
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0  
          TX packets:12 errors:0 dropped:0 overruns:0 carrier:0  
          collisions:0 txqueuelen:1000   
          RX bytes:0 (0.0 b)  TX bytes:816 (816.0 b)  
  
lo        Link encap:Local Loopback    
          inet addr:127.0.0.1  Mask:255.0.0.0  
          inet6 addr: ::1/128 Scope:Host  
          UP LOOPBACK RUNNING  MTU:16436  Metric:1  
          RX packets:114 errors:0 dropped:0 overruns:0 frame:0  
          TX packets:114 errors:0 dropped:0 overruns:0 carrier:0  
          collisions:0 txqueuelen:0   
          RX bytes:9974 (9.7 KiB)  TX bytes:9974 (9.7 KiB)  
  
13.修改配置rac01,rac02两台机器的hosts文件  
rac01-> cat /etc/hosts  
127.0.0.1       localhost localhost.localdomain localhost4 localhost4.localdomain4  
\#::1         localhost localhost.localdomain localhost6 localhost6.localdomain6  
10.38.1.182     rac01  
10.38.1.85      rac02  
\#private ip  
192.168.56.102  rac01-priv  
192.168.56.101  rac02-priv  
\#public ip  
10.38.1.56      rac01-vip  
10.38.1.59      rac02-vip  
\#scan ip  
10.38.1.188     scanip  
  
rac02-> cat /etc/hosts  
\#127.0.0.1      lcalhost localhost.localdomain localhost4 localhost4.localdomain4  
\#::1         localhost localhost.localdomain localhost6 localhost6.localdomain6  
127.0.0.1 localhost.localdomain localhost  
10.38.1.182     rac01  
10.38.1.85      rac02  
\#private ip  
192.168.56.102       rac01-priv  
192.168.56.101       rac02-priv  
\#public ip  
10.38.1.56     rac01-vip  
10.38.1.59     rac02-vip  
\#scan ip  
10.38.1.188     scanip  
  
14.虚拟机添加共享磁盘  
VBoxManage createhd --filename E:\\Oracleracvm\\share.vdi --size 30000 --format VDI --variant Fixed  
VBoxManage createhd --filename E:\\Oracleracvm\\share01.vdi --size 1000 --format VDI --variant Fixed  
VBoxManage createhd --filename E:\\Oracleracvm\\share02.vdi --size 2000 --format VDI --variant Fixed  
添加共享磁盘到rac01,rac02节点  
参加：  
http://blog.csdn.net/xcl168/article/details/17539663  
https://jingyan.baidu.com/article/0a52e3f4154b11bf62ed72ad.html --图形界面添加共享磁盘  
  
15.UDEV绑定共享存储  
\[root@rac02\]\# /sbin/scsi\_id --whitelisted --replace-whitespace --device=/dev/sdb  
1ATA\_VBOX\_HARDDISK\_VB931df84b-31600e36  
\[root@rac02\]\# /sbin/scsi\_id --whitelisted --replace-whitespace --device=/dev/sdc  
1ATA\_VBOX\_HARDDISK\_VBc84e4363-ce0b3691  
\[root@rac02\]\# /sbin/scsi\_id --whitelisted --replace-whitespace --device=/dev/sdd  
1ATA\_VBOX\_HARDDISK\_VB0c618945-fab402cb  
\[root@rac02 ~\]\# cd /etc/udev/rules.d/  
\[root@rac02 rules.d\]\# ls  
60-fprint-autosuspend.rules  70-persistent-cd.rules   90-hal.rules  
60-pcmcia.rules              70-persistent-net.rules  98-kexec.rules  
60-raw.rules                 90-alsa.rules            99-fuse.rules  
\[root@rac02 rules.d\]\# vi 99-oracle-asmdevices.rules  
KERNEL=="sdb", BUS=="scsi", PROGRAM=="/sbin/scsi\_id -gud /dev/$name" RESULT=="1ATA\_VBOX\_HARDDISK\_VB931df84b-31600e36", SYMLINK+="asmdisk/asm-data",  OWNER="grid", GROUP="asmadmin", MODE="0660"  
KERNEL=="sdc", BUS=="scsi", PROGRAM=="/sbin/scsi\_id -gud /dev/$name" RESULT=="1ATA\_VBOX\_HARDDISK\_VBc84e4363-ce0b3691", SYMLINK+="asmdisk/asm-crs",  OWNER="grid", GROUP="asmadmin", MODE="0660"  
KERNEL=="sdd", BUS=="scsi", PROGRAM=="/sbin/scsi\_id -gud /dev/$name" RESULT=="1ATA\_VBOX\_HARDDISK\_VB0c618945-fab402cb", SYMLINK+="asmdisk/asm-voting",  OWNER="grid", GROUP="asmadmin", MODE="0660"  
  
重启UDEV:  
\[root@cl6-11gr2-rac2 ~\]\# /sbin/start\_udev   
查看绑定后的共享存储:  
\[root@cl6-11gr2-rac2 ~\]\# ls -ltr /dev/asm\*  
brw-rw----. 1 grid asmadmin 8, 32 8月  17 15:27 /dev/asm-diskc  
brw-rw----. 1 grid asmadmin 8, 16 8月  17 15:27 /dev/asm-diskb  
brw-rw----. 1 grid asmadmin 8, 48 8月  17 15:27 /dev/asm-diskd  
参考：  
https://www.cnblogs.com/lanston/p/3917936.html  
http://blog.sina.com.cn/s/blog\_8317516b0101agp5.html  
  
16.安装grid  
解压p13390677\_112040\_Linux-x86-64\_3of7  
以grid用户登录执行./runInstaller,启动安装界面  
（用xmanager中xstart来设置启动安装界面）参见 http://blog.csdn.net/weiwangsisoftstone/article/details/51782739  
图形界面一步一步安装最后执行脚本  
\[root@rac01 tmp\]\# /u01/app/11.2.0/grid/root.sh   
Performing root user operation for Oracle 11g   
  
The following environment variables are set as:  
    ORACLE\_OWNER= grid  
    ORACLE\_HOME=  /u01/app/11.2.0/grid  
  
Enter the full pathname of the local bin directory: \[/usr/local/bin\]:   
   Copying dbhome to /usr/local/bin ...  
   Copying oraenv to /usr/local/bin ...  
   Copying coraenv to /usr/local/bin ...  
\[root@rac01 tmp\]\# /u01/app/11.2.0/grid/root.sh  
Performing root user operation for Oracle 11g   
  
The following environment variables are set as:  
    ORACLE\_OWNER= grid  
    ORACLE\_HOME=  /u01/app/11.2.0/grid  
  
Enter the full pathname of the local bin directory: \[/usr/local/bin\]:   
The contents of "dbhome" have not changed. No need to overwrite.  
The contents of "oraenv" have not changed. No need to overwrite.  
The contents of "coraenv" have not changed. No need to overwrite.  
  
Entries will be added to the /etc/oratab file as needed by  
Database Configuration Assistant when a database is created  
Finished running generic part of root script.  
Now product-specific root actions will be performed.  
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig\_params  
User ignored Prerequisites during installation  
Installing Trace File Analyzer  
OLR initialization - successful  
  root wallet  
  root wallet cert  
  root cert export  
  peer wallet  
  profile reader wallet  
  pa wallet  
  peer wallet keys  
  pa wallet keys  
  peer cert request  
  pa cert request  
  peer cert  
  pa cert  
  peer root cert TP  
  profile reader root cert TP  
  pa root cert TP  
  peer pa cert TP  
  pa peer cert TP  
  profile reader pa cert TP  
  profile reader peer cert TP  
  peer user cert  
  pa user cert  
Adding Clusterware entries to upstart  
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac01'  
CRS-2676: Start of 'ora.mdnsd' on 'rac01' succeeded  
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac01'  
CRS-2676: Start of 'ora.gpnpd' on 'rac01' succeeded  
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac01'  
CRS-2672: Attempting to start 'ora.gipcd' on 'rac01'  
CRS-2676: Start of 'ora.cssdmonitor' on 'rac01' succeeded  
CRS-2676: Start of 'ora.gipcd' on 'rac01' succeeded  
CRS-2672: Attempting to start 'ora.cssd' on 'rac01'  
CRS-2672: Attempting to start 'ora.diskmon' on 'rac01'  
CRS-2676: Start of 'ora.diskmon' on 'rac01' succeeded  
CRS-2676: Start of 'ora.cssd' on 'rac01' succeeded  
宸叉寤哄苟邋?ASM?  
宸叉寤虹ｇ缁  
            OCR?  
clscfg: -install mode specified  
Successfully accumulated necessary OCR keys.  
Creating OCR keys for user 'root', privgrp 'root'..  
Operation successful.  
CRS-4256: Updating the profile  
Successful addition of voting disk 29b6c5afb5064fb8bfbec20c120aa02b.  
Successfully replaced voting disk group with +OCR.  
CRS-4256: Updating the profile  
CRS-4266: Voting file(s) successfully replaced  
\#\#  STATE    File Universal Id                File Name Disk group  
\--  -----    -----------------                --------- ---------  
 1. ONLINE   29b6c5afb5064fb8bfbec20c120aa02b (/dev/asmdisk/asm-crs) \[OCR\]  
Located 1 voting disk(s).  
CRS-2672: Attempting to start 'ora.OCR.dg' on 'rac01'  
CRS-2676: Start of 'ora.OCR.dg' on 'rac01' succeeded  
Configure Oracle Grid Infrastructure for a Cluster ... succeeded  
\[root@rac01 tmp\]\#   
  
17.安装oracle  
解压  
p13390677\_112040\_Linux-x86-64\_1of7  
p13390677\_112040\_Linux-x86-64\_2of7  
以oracle用户登录执行./runInstaller  
最后安装执行脚本root.sh  
  
18.创建数据库  
以oracle用户登录到任何一个节点上执行dbca创建数据库  
  
至此按照完毕  
rac资源状态  
\[root@rac01 oraInventory\]\# su - grid  
rac01-> crsctl status res -t  
\--------------------------------------------------------------------------------  
NAME           TARGET  STATE        SERVER                   STATE\_DETAILS         
\--------------------------------------------------------------------------------  
Local Resources  
\--------------------------------------------------------------------------------  
ora.DATA.dg  
               ONLINE  ONLINE       rac01                                          
               ONLINE  ONLINE       rac02                                          
ora.LISTENER.lsnr  
               ONLINE  ONLINE       rac01                                          
               ONLINE  ONLINE       rac02                                          
ora.OCR.dg  
               ONLINE  ONLINE       rac01                                          
               ONLINE  ONLINE       rac02                                          
ora.asm  
               ONLINE  ONLINE       rac01                    Started               
               ONLINE  ONLINE       rac02                    Started               
ora.gsd  
               OFFLINE OFFLINE      rac01                                          
               OFFLINE OFFLINE      rac02                                          
ora.net1.network  
               ONLINE  ONLINE       rac01                                          
               ONLINE  ONLINE       rac02                                          
ora.ons  
               ONLINE  ONLINE       rac01                                          
               ONLINE  ONLINE       rac02                                          
\--------------------------------------------------------------------------------  
Cluster Resources  
\--------------------------------------------------------------------------------  
ora.LISTENER\_SCAN1.lsnr  
      1        ONLINE  ONLINE       rac01                                          
ora.cvu  
      1        ONLINE  ONLINE       rac01                                          
ora.oc4j  
      1        ONLINE  ONLINE       rac01                                          
ora.rac01.vip  
      1        ONLINE  ONLINE       rac01                                          
ora.rac02.vip  
      1        ONLINE  ONLINE       rac02                                          
ora.rac11g.db  
      1        ONLINE  ONLINE       rac01                    Open                  
      2        ONLINE  ONLINE       rac02                    Open                  
ora.scan1.vip  
      1        ONLINE  ONLINE       rac01                                          
rac01->   
  
  
安装遇到问题以及解决：  
  
执行/u01/app/11.2.0/grid/root.sh 脚本报错  
\[root@rac01 tmp\]\# /u01/app/11.2.0/grid/root.sh   
Performing root user operation for Oracle 11g   
The following environment variables are set as:  
    ORACLE\_OWNER= grid  
    ORACLE\_HOME=  /u01/app/11.2.0/grid  
Enter the full pathname of the local bin directory: \[/usr/local/bin\]:   
   Copying dbhome to /usr/local/bin ...  
   Copying oraenv to /usr/local/bin ...  
   Copying coraenv to /usr/local/bin ...  
  
Creating /etc/oratab file...  
Entries will be added to the /etc/oratab file as needed by  
Database Configuration Assistant when a database is created  
Finished running generic part of root script.  
Now product-specific root actions will be performed.  
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig\_params  
Creating trace directory  
User ignored Prerequisites during installation  
Installing Trace File Analyzer  
Failed to create keys in the OLR, rc = 127, Message:  
  /u01/app/11.2.0/grid/bin/clscfg.bin: error while loading shared libraries: libcap.so.1: cannot open shared object file: No such file or directory   
  
Failed to create keys in the OLR at /u01/app/11.2.0/grid/crs/install/crsconfig\_lib.pm line 7660.  
/u01/app/11.2.0/grid/perl/bin/perl -I/u01/app/11.2.0/grid/perl/lib -I/u01/app/11.2.0/grid/crs/install /u01/app/11.2.0/grid/crs/install/rootcrs.pl execution failed  
Failed to create keys in the OLR, rc = 127, Message:  
/opt/app/11.2.0/grid/bin/clscfg.bin: error while loading shared libraries: libcap.so.1: cannot open shared object file: No such file or directory  
  
执行如下操作，就可以在此执行root.sh了  
\[root@rac1 ~\]\# cd /lib64  
\[root@rac1 lib64\]\# ls libcap\*  
libcap-ng.so.0 libcap-ng.so.0.0.0 libcap.so.2 libcap.so.2.16  
\[root@rac1 lib64\]\# ln -s libcap.so.2.16 libcap.so.1  
\[root@rac2 lib64\]\# ln -s libcap.so.2.16 libcap.so.1  
  
2.节点的vip被占用导致节点vip资源offline无法启动  
  
修改节点hosts文件和集群vip  
rac02->  srvctl config vip -n rac02  
VIP exists: /rac02-vip/10.38.1.57/10.38.1.0/255.255.255.0/eth0, hosting node rac02  
rac02->  srvctl config vip -n rac01  
VIP exists: /rac01-vip/10.38.1.56/10.38.1.0/255.255.255.0/eth0, hosting node rac01  
srvctl modify nodeapps -n rac02 -A 10.38.1.59/255.255.255.0/eth0  
重启vip资源问题解决  
  
3.ONS服务offline 启动报错  
rac02-> onsctl start  
CRS-2672: Attempting to start 'ora.ons' on 'rac02'  
CRS-5016: Process "/u01/app/11.2.0/grid/opmn/bin/onsctli" spawned by agent "/u01/app/11.2.0/grid/bin/oraagent.bin" for action "start" failed: details at "(:CLSN00010:)" in "/u01/app/11.2.0/grid/log/rac02/agent/crsd/oraagent\_grid/oraagent\_grid.log"  
CRS-2674: Start of 'ora.ons' on 'rac02' failed  
CRS-4000: Command Start failed, or completed with errors.  
  
  
解决方法：  
CRS-2799: Failed to start or stop resource 'ora.ons' on 'node1' due to incorrect hosts configuration (CRS-2674: Start of 'ora.ons' on '<host>' failed) (文档 ID 1926115.1) 转到底部转到底部   
In this Document  
Symptoms  
Cause  
Solution  
References  
APPLIES TO:  
Oracle Database - Enterprise Edition - Version 11.2.0.1 and later  
Information in this document applies to any platform.  
SYMPTOMS  
Unable to start or stop 11.2 GRID Clusterware on one node of a two node cluster.  
Errors on starting:  
CRS-2674: Start of 'ora.ons' on '<host>' failed  
Errors on stopping:  
CRS-5014: Agent "/u01/grid/oracle/product/11.2.0.4/grid\_source/bin/oraagent.bin" timed out starting process "/u01/grid/oracle/product/11.2.0.4/grid\_source/opmn/bin/onsctli" for action "clean": details at "(:CLSN00009:)" in "/u01/grid/oracle/product/11.2.0.4/grid\_source/log/node1/agent/crsd/oraagent\_oracle/oraagent\_oracle.log"  
CRS-5017: The resource action "ora.ons clean" encountered the following error:  
(:CLSN00009:)Utils:execCmd aborted. For details refer to "(:CLSN00106:)" in "/u01/grid/oracle/product/11.2.0.4/grid\_source/log/node1/agent/crsd/oraagent\_oracle/oraagent\_oracle.log".  
CRS-5014: Agent "/u01/grid/oracle/product/11.2.0.4/grid\_source/bin/oraagent.bin" timed out starting process "/u01/grid/oracle/product/11.2.0.4/grid\_source/opmn/bin/onsctli" for action "check": details at "(:CLSN00009:)" in "/u01/grid/oracle/product/11.2.0.4/grid\_source/log/node1/agent/crsd/oraagent\_oracle/oraagent\_oracle.log"  
CRS-5017: The resource action "ora.ons check" encountered the following error:  
(:CLSN00009:)Utils:execCmd aborted. For details refer to "(:CLSN00109:)" in "/u01/grid/oracle/product/11.2.0.4/grid\_source/log/node1/agent/crsd/oraagent\_oracle/oraagent\_oracle.log".  
CRS-2680: Clean of 'ora.ons' on 'node1' failed  
CRS-2799: Failed to shut down resource 'ora.node1.ons' on 'ctdb1'  
CRS-2799: Failed to shut down resource 'ora.ons' on 'node1'  
CRS-2794: Shutdown of Cluster Ready Services-managed resources on 'node1' has failed  
ons.log.node1  
2014-04-11T16:39:25-05:00\] \[ons\] \[NOTIFICATION:1\] \[104\] \[ons-internal\] ONS server initiated  
\[2014-04-11T16:39:25-05:00\] \[ons\] \[ERROR:1\] \[17\] \[ons-listener\] 10.x.x.x,6100: BIND (Can't assign requested address)  
\[2014-04-11T16:39:48-05:00\] \[ons\] \[NOTIFICATION:1\] \[104\] \[ons-internal\] ONS server initiated  
\[2014-04-11T16:39:48-05:00\] \[ons\] \[ERROR:1\] \[17\] \[ons-listener\] 10.x.x.x,6100: BIND (Can't assign requested address)  
\[2014-04-11T16:41:20-05:00\] \[ons\] \[NOTIFICATION:1\] \[104\] \[ons-internal\] ONS server initiated  
\[2014-04-11T16:41:20-05:00\] \[ons\] \[ERROR:1\] \[17\] \[ons-listener\] 10.x.x.x,6100: BIND (Can't assign requested address)  
   
CAUSE  
ping localhost.<domain> was returning the non-existence IP address. DNS was picking that IP resulting ONS start/stop failure.  
node1\# ping localhost.<domain>  
PING <domain>: 64 byte packets  
64 bytes from 10.x.x.x: icmp\_seq=0. time=5. ms  
64 bytes from 10.x.x.x: icmp\_seq=0. time=5. ms  
64 bytes from 10.x.x.x: icmp\_seq=0. time=5. ms  
3 packets transmitted, 3 packets received, 0% packet loss  
round-trip (ms)  min/avg/max = 1/2/5  
node1\# ping localhost                          
PING cern-mt015672.northamerica.cerner.net: 64 byte packets  
64 bytes from 10.x.x.x: icmp\_seq=0. time=5. ms  
64 bytes from 10.x.x.x: icmp\_seq=0. time=5. ms  
64 bytes from 10.x.x.x: icmp\_seq=0. time=5. ms  
  
  
SOLUTION  
1.  Update DNS to remove the reference of non-existent ip address of localhost.<domain> in DNS. This fixed the issue of ONS stop/start.  
  
2.  Alternatively or perhaps short team,  host name resolution may be performed using local /etc/hosts rather than DNS.  
Example for AIX platform:  
Edit /etc/netsvc.conf, add:  
hosts = local, bind  
Example for Linux platform:  
Edit /etc/nsswitch.conf:  
hosts: files dns  
Ensure the following is set in /etc/hosts:  
127.0.0.1 localhost.localdomain localhost  
Finally, ping localhost to verify 127.0.0.1 is returned.  After this ora.ons should start and stop without error.  
  
4.grid安装报\[INS-20802\] Oracle Cluster Verification Utility failed.  
  
导致这个错误的原因是在/etc/hosts中配置了SCAN的地址，尝试ping这个地址信息，如果可以成功，则这个错误可以忽略。  
我尝试ping scan ip可以ping通，所以暂时也就忽略了这个错误。