Redhat 5.4 RAC multipath 配置raw，运行root.sh 时报错Failed to upgrade Oracle Cluster Registry configuration 解决方法

之前打算装Oracle 11gR2的RAC的。 后来因为资源不够，就改装oracle 10g了。

raw 是用openfiler 配置 加 Multipath多路径配置的。

raw的配置参考我的Blog：

Linux 5.4 LVM RAW 设备 配置的深入研究

http://blog.csdn.net/tianlesoftware/archive/2010/11/26/6036263.aspx

这种情况下配置有些注意事项。 否则会在执行root.sh命令的时候，报错：

[root@rac1 init.d]# /u01/app/oracle/product/crs/root.sh

WARNING: directory '/u01/app/oracle/product' is not owned by root

WARNING: directory '/u01/app/oracle' is not owned by root

WARNING: directory '/u01/app' is not owned by root

WARNING: directory '/u01' is not owned by root

Checking to see if Oracle CRS stack is already configured

Setting the permissions on OCR backup directory

Setting up NS directories

PROT-1: Failed to initialize ocrconfig

Failed to upgrade Oracle Cluster Registry configuration

一般出现这个问题的原因有几种：

（1） Bug

这种情况参考：

Executing root.sh errors with "Failed To Upgrade Oracle Cluster Registry Configuration" [ID 466673.1]

http://blog.csdn.net/tianlesoftware/archive/2010/11/26/6037141.aspx

（2） raw 设备的权限问题

[root@rac1 raw]# pwd

/dev/raw

[root@rac1 raw]# ll

total 0

crw-r----- 1 root oinstall 162, 1 Nov 25 15:34 raw1

crw-r----- 1 root oinstall 162, 2 Nov 25 15:34 raw2

crw-r--r-- 1 oracle oinstall 162, 3 Nov 25 13:48 raw3

crw-r--r-- 1 oracle oinstall 162, 4 Nov 25 13:48 raw4

crw-r--r-- 1 oracle oinstall 162, 5 Nov 25 13:48 raw5

权限要设置成如下：

chown root.oinstall /dev/raw/raw[1-2]
chown oracle.oinstall /dev/raw/raw[3-5]


chmod 775 /dev/raw/raw[1-2]
chmod 775 /dev/raw/raw[3-5]

可以在udev的规则里设置， 也可以把权限添加到/etc/rc.d/rc.local 中。

（3） raw 设备读写问题

这种情况参考：

Placement of Voting disk and OCR Files in Oracle RAC 10g and 11gR1 [ID 293819.1]

http://blog.csdn.net/tianlesoftware/archive/2010/11/26/6037108.aspx

清空一下RAW:

[root@rac1 raw]# dd if=/dev/zero of=/dev/mapper/crsp6 bs=1M count=200

dd: writing `/dev/mapper/crsp6': No space left on device

197+0 records in

196+0 records out

205599744 bytes (206 MB) copied, 21.1817 seconds, 9.7 MB/s

说明：

这里有个提示信息，磁盘空间不足: No space left on device。 这里的情况是， 我的每个raw是200M。 但是实际容量并没有这么多。 所以我清空写200M的时候，会报空间不足。 这种情况会按照实际大小进行清空。 从下面我们可以该raw设备实际大小是197M。

[root@rac1 raw]# dd if=/dev/zero of=/dev/mapper/crsp6 bs=1M count=190

190+0 records in

190+0 records out

199229440 bytes (199 MB) copied, 21.9521 seconds, 9.1 MB/s

[root@rac1 raw]# dd if=/dev/zero of=/dev/mapper/crsp5 bs=1M count=195

195+0 records in

195+0 records out

204472320 bytes (204 MB) copied, 22.0362 seconds, 9.3 MB/s

[root@rac1 raw]# dd if=/dev/zero of=/dev/mapper/crsp7 bs=1M count=195

195+0 records in

195+0 records out

204472320 bytes (204 MB) copied, 19.9692 seconds, 10.2 MB/s

[root@rac1 raw]# dd if=/dev/zero of=/dev/mapper/crsp8 bs=1M count=195

195+0 records in

195+0 records out

204472320 bytes (204 MB) copied, 18.5526 seconds, 11.0 MB/s

[root@rac1 raw]# dd if=/dev/zero of=/dev/mapper/crsp9 bs=1M count=195

195+0 records in

195+0 records out

204472320 bytes (204 MB) copied, 21.9439 seconds, 9.3 MB/s

我这次遇到问题的Log如下：

[root@rac1 client]# pwd

/u01/app/oracle/product/crs/log/rac1/client

[root@rac1 client]# ls

ocrconfig_7527.log

[root@rac1 client]# cat ocrconfig_7527.log

Oracle Database 10g CRS Release 10.2.0.1.0 Production Copyright 1996, 2005 Oracle. All rights reserved.

2010-11-25 19:54:14.694: [ OCRCONF][1189568]ocrconfig starts...

2010-11-25 19:54:14.696: [ OCRCONF][1189568]Upgrading OCR data

2010-11-25 19:54:15.434: [ OCRRAW][1189568]propriogid:1: INVALID FORMAT

2010-11-25 19:54:15.464: [ OCRRAW][1189568]propriogid:1: INVALID FORMAT

2010-11-25 19:54:15.464: [ OCRRAW][1189568]proprioini: both disks are not OCR formatted

2010-11-25 19:54:15.464: [ OCRRAW][1189568]proprinit: Could not open raw device

2010-11-25 19:54:15.465: [ default][1189568]a_init:7!: Backend init unsuccessful : [26]

2010-11-25 19:54:15.465: [ OCRCONF][1189568]Exporting OCR data to [OCRUPGRADEFILE]

2010-11-25 19:54:15.465: [ OCRAPI][1189568]a_init:7!: Backend init unsuccessful : [33]

2010-11-25 19:54:15.465: [ OCRCONF][1189568]There was no previous version of OCR. error:[PROC-33: Oracle Cluster Registry is not configured]

2010-11-25 19:54:16.072: [ OCRRAW][1189568]propriogid:1: INVALID FORMAT

2010-11-25 19:54:16.075: [ OCRRAW][1189568]propriogid:1: INVALID FORMAT

2010-11-25 19:54:16.075: [ OCRRAW][1189568]proprioini: both disks are not OCR formatted

2010-11-25 19:54:16.075: [ OCRRAW][1189568]proprinit: Could not open raw device

2010-11-25 19:54:16.076: [ default][1189568]a_init:7!: Backend init unsuccessful : [26]

2010-11-25 19:54:16.231: [ OCRRAW][1189568]propriogid:1: INVALID FORMAT

2010-11-25 19:54:16.232: [ OCRRAW][1189568]propriogid:1: INVALID FORMAT

2010-11-25 19:54:16.265: [ OCRRAW][1189568]ibctx:1:ERROR: INVALID FORMAT

2010-11-25 19:54:16.265: [ OCRRAW][1189568]proprinit:problem reading the bootblock or superbloc 22

2010-11-25 19:54:16.646: [ OCRRAW][1189568]propriogid:1: INVALID FORMAT

2010-11-25 19:54:16.647: [ OCRRAW][1189568]propriogid:1: INVALID FORMAT

2010-11-25 19:54:16.757: [ OCRRAW][1189568]propriowv: Vote information on disk 0 [/dev/raw/raw1] is adjusted from [0/0] to [1/2]

2010-11-25 19:54:16.762: [ OCRRAW][1189568]propriowv: Vote information on disk 1 [/dev/raw/raw2] is adjusted from [0/0] to [1/2]

2010-11-25 19:54:17.193: [ OCRRAW][1189568]propriniconfig:No 92 configuration

2010-11-25 19:54:17.193: [ OCRAPI][1189568]a_init:6a: Backend init successful

2010-11-25 19:54:18.263: [ OCRCONF][1189568]Initialized DATABASE keys in OCR

2010-11-25 19:54:18.478: [ OCRCONF][1189568]csetskgfrblock0: clsfmt returned with error [4].

2010-11-25 19:54:18.478: [ OCRCONF][1189568]Failure in setting block0 [-1]

2010-11-25 19:54:18.478: [ OCRCONF][1189568]OCR block 0 is not set !

2010-11-25 19:54:18.479: [ OCRCONF][1189568]Exiting [status=failed]...

这个是Multipath 的bug 问题，就是我们遇到的第一种情况：

（1） 用clsfmt检查raw 设备

[root@rac1 bin]# ./clsfmt.bin ocr /dev/raw/raw1

clsfmt: Failure trying to resize OCR file from 205567488 bytes to 205570048 bytes

clsfmt: Received unexpected error 4 from skgfifi

skgfifi: Additional information: -2

Additional information: 205567488

这里没有成功， 就是bug。

（2）用Pathch中的新文件替换clsfmt.bin 文件。

Patch 是：p4679769。 这个需要Oracle 帐号才能下载。 所以我将Linux 平台下32位和64位的path都上传到csdn的下载里了。 有遇到相同问题的朋友自己去下载。 下载地址：

http://download.csdn.net/source/2862150

（3）再次测试：

[root@rac1 bin]# ./clsfmt ocr /dev/raw/raw1

clsfmt: Failure trying to resize OCR file from 205567488 bytes to 205570048 bytes

clsfmt: successfully initialized file /dev/raw/raw1

[root@rac1 bin]# ./clsfmt ocr /dev/raw/raw2

clsfmt: Failure trying to resize OCR file from 205599744 bytes to 205602816 bytes

clsfmt: successfully initialized file /dev/raw/raw2

[root@rac1 bin]# ./clsfmt ocr /dev/raw/raw3

clsfmt: Failure trying to resize OCR file from 205599744 bytes to 205602816 bytes

clsfmt: successfully initialized file /dev/raw/raw3

[root@rac1 bin]# ./clsfmt ocr /dev/raw/raw4

clsfmt: Failure trying to resize OCR file from 205599744 bytes to 205602816 bytes

clsfmt: successfully initialized file /dev/raw/raw4

[root@rac1 bin]# ./clsfmt ocr /dev/raw/raw5

clsfmt: Failure trying to resize OCR file from 205599744 bytes to 205602816 bytes

clsfmt: successfully initialized file /dev/raw/raw5

bug 修复成功了。

（4） 清空raw设备

因为我们已经执行过root.sh 命令，会有相关信息写到raw设备里。 用dd命令清空这些信息。 最好留一点空间。 因为在Solaris平台上有预留 1M 空间的做法，不然会出现问题。

[root@rac1 rac1]# dd if=/dev/zero of=/dev/raw/raw1 bs=10M count=10

10+0 records in

10+0 records out

104857600 bytes (105 MB) copied, 17.4534 seconds, 6.0 MB/s

[root@rac1 rac1]# dd if=/dev/zero of=/dev/raw/raw2 bs=10M count=10

10+0 records in

10+0 records out

104857600 bytes (105 MB) copied, 11.4307 seconds, 9.2 MB/s

[root@rac1 rac1]# dd if=/dev/zero of=/dev/raw/raw3 bs=10M count=10

10+0 records in

10+0 records out

104857600 bytes (105 MB) copied, 18.4136 seconds, 5.7 MB/s

[root@rac1 rac1]# dd if=/dev/zero of=/dev/raw/raw4 bs=10M count=10

10+0 records in

10+0 records out

104857600 bytes (105 MB) copied, 17.821 seconds, 5.9 MB/s

[root@rac1 rac1]# dd if=/dev/zero of=/dev/raw/raw5 bs=10M count=10

10+0 records in

10+0 records out

104857600 bytes (105 MB) copied, 17.3289 seconds, 6.1 MB/s

如果不清空raw 设备，直接执行root.sh 命令，还是会报错误：

[root@rac1 bin]# /u01/app/oracle/product/crs/root.sh

WARNING: directory '/u01/app/oracle/product' is not owned by root

WARNING: directory '/u01/app/oracle' is not owned by root

WARNING: directory '/u01/app' is not owned by root

WARNING: directory '/u01' is not owned by root

Checking to see if Oracle CRS stack is already configured

Setting the permissions on OCR backup directory

Setting up NS directories

PROT-1: Failed to initialize ocrconfig

Failed to upgrade Oracle Cluster Registry configuration

错误的log 日志如下：

#cat /u01/app/oracle/product/crs/log/rac1/client/ocrconfig_16159.log

Oracle Database 10g CRS Release 10.2.0.1.0 Production Copyright 1996, 2005 Oracle. All rights reserved.

2010-11-25 23:27:25.468: [ OCRCONF][2118112]ocrconfig starts...

2010-11-25 23:27:25.469: [ OCRCONF][2118112]Upgrading OCR data

2010-11-25 23:27:25.558: [ OCRRAW][2118112]propriogid:1: INVALID FORMAT

2010-11-25 23:27:25.559: [ OCRRAW][2118112]proprioini: disk 0 (/dev/raw/raw1) doesn't have enough votes (1,2)

2010-11-25 23:27:25.559: [ OCRRAW][2118112]proprinit: Could not open raw device

2010-11-25 23:27:25.560: [ default][2118112]a_init:7!: Backend init unsuccessful : [26]

2010-11-25 23:27:25.560: [ OCRCONF][2118112]Exporting OCR data to [OCRUPGRADEFILE]

2010-11-25 23:27:25.560: [ OCRAPI][2118112]a_init:7!: Backend init unsuccessful : [33]

2010-11-25 23:27:25.560: [ OCRCONF][2118112]There was no previous version of OCR. error:[PROC-33: Oracle Cluster Registry is not configured]

2010-11-25 23:27:25.746: [ OCRRAW][2118112]propriogid:1: INVALID FORMAT

2010-11-25 23:27:25.746: [ OCRRAW][2118112]proprioini: disk 0 (/dev/raw/raw1) doesn't have enough votes (1,2)

2010-11-25 23:27:25.746: [ OCRRAW][2118112]proprinit: Could not open raw device

2010-11-25 23:27:25.747: [ default][2118112]a_init:7!: Backend init unsuccessful : [26]

2010-11-25 23:27:26.112: [ OCRRAW][2118112]propriogid:1: INVALID FORMAT

2010-11-25 23:27:26.112: [ OCRRAW][2118112]proprioini: disk 0 (/dev/raw/raw1) doesn't have enough votes (1,2)

2010-11-25 23:27:26.112: [ OCRRAW][2118112]proprinit: Could not open raw device

2010-11-25 23:27:26.120: [ OCRAPI][2118112]a_init:6b!: Backend init unsuccessful : [26]

2010-11-25 23:27:26.120: [ OCRCONF][2118112]Failed to initialized OCR context. error:[PROC-26: Error while accessing the physical storage]

2010-11-25 23:27:26.120: [ OCRCONF][2118112]Exiting [status=failed]...

（5）清空raw 设备后，在执行/u01/app/oracle/product/crs/root.sh 命令

这次成功执行。

------------------------------------------------------------------------------

Blog： http://blog.csdn.net/tianlesoftware

网上资源： http://tianlesoftware.download.csdn.net

相关视频：http://blog.csdn.net/tianlesoftware/archive/2009/11/27/4886500.aspx

DBA1 群：62697716(满); DBA2 群：62697977(满)

DBA3 群：62697850 DBA 超级群：63306533;

聊天 群：40132017

--加群需要在备注说明Oracle表空间和数据文件的关系，否则拒绝申请