OCR Internals

With Oracle Grid Infrastructure 11gr2, Oracle ASM and Oracle Clusterware are installed into a single home directory known as Grid Infrastructure Home.
Oracle Clusterware consists of 2 stacks:-
1) The Cluster Ready Service Stack ( CRS Stack ) :- It manages (start, stop, monitor, and failover operations) the cluster resources based on the configuration information stored in OCR for each resource.
2) The Oracle High Availability Service Stack:- It is responsible for monitoring and maintaining high availability of Oracle ASM and Clusterware itself.

The Oracle Cluster Registry (OCR) file is a key component of Oracle Clusterware. It maintains information about the high-availability components in your cluster, such as the voting file details, cluster database instance to node mapping, and CRS application resource profiles (such as services, Virtual Interconnect Protocol addresses, and so on),

In this post we’ll see what does an OCR contains, what information does OCR have, OCR’s purpose, who updates OCR, who maintains OCR etc. To understand all these questions we need to understand what it contains, hence taking a peek into OCR backup.

For the sake of simplicity would only discuss about the main components from the dump of OCR, Please Click OCR Dump if you want to see the complete dump.

[SYSTEM]
[SYSTEM.version]
[SYSTEM.version.activeversion]
[SYSTEM.version.hostnames]
[SYSTEM.version.hostnames.rac1]
[SYSTEM.version.hostnames.rac2]
[SYSTEM.versionstring]
[SYSTEM.WALLET]
[SYSTEM.GNS]
[SYSTEM.css]
[SYSTEM.css.interfaces]
[SYSTEM.ORA_CRS_HOME]
[SYSTEM.evm]
[SYSTEM.GPnP.profiles]
[SYSTEM.CRSADMIN]
[SYSTEM.CRSUSER]
[SYSTEM.CRSD]
[SYSTEM.CRSD.SERVERPOOLS]
[SYSTEM.CRSD.SERVERS]
[SYSTEM.CRSD.SERVERS.rac1.STATE]
[SYSTEM.CRSD.TYPES]
[SYSTEM.CRSD.TYPES.ora!local_resource!type.START_DEPENDENCIES]
[SYSTEM.CRSD.TYPES.ora!local_resource!type.START_DEPENDENCIES.CONFIG]
[SYSTEM.CRSD.TYPES.ora!local_resource!type.STOP_DEPENDENCIES]
[SYSTEM.CRSD.TYPES.ora!local_resource!type.STOP_DEPENDENCIES.CONFIG]
[SYSTEM.CRSD.TYPES.ora!network!type]
[SYSTEM.CRSD.TYPES.ora!network!type.DESCRIPTION]
[SYSTEM.CRSD.TYPES.ora!cluster_resource!type.AUTO_START]
[SYSTEM.CRSD.TYPES.ora!cluster_resource!type.DESCRIPTION]
[SYSTEM.CRSD.TYPES.ora!cluster_resource!type.SERVER_POOLS]
[SYSTEM.CRSD.TYPES.ora!cluster_vip!type]
[SYSTEM.CRSD.TYPES.ora!gsd!type]
[SYSTEM.CRSD.TYPES.ora!ons!type]
[SYSTEM.CRSD.TYPES.ora!asm!type.START_TIMEOUT]
[SYSTEM.CRSD.TYPES.ora!asm!type.STOP_TIMEOUT]
[SYSTEM.CRSD.TYPES.ora!asm!type.USR_ORA_STOP_MODE]
[SYSTEM.CRSD.TYPES.ora!diskgroup!type.START_DEPENDENCIES]
[SYSTEM.CRSD.TYPES.ora!diskgroup!type.START_TIMEOUT]
[SYSTEM.CRSD.TYPES.ora!diskgroup!type.USR_ORA_STOP_MODE.CONFIG]
[SYSTEM.CRSD.TYPES.ora!scan_listener!type.CONFIG_VERSION.CONFIG]
[SYSTEM.CRSD.TYPES.ora!listener!type.TYPE_ACL]
[SYSTEM.CRSD.TYPES.ora!listener!type.USR_ORA_OPI]
[SYSTEM.CRSD.TYPES.ora!database!type.BASE_TYPE]
[SYSTEM.CRSD.TYPES.ora!database!type.CHECK_TIMEOUT]
[SYSTEM.CRSD.TYPES.ora!database!type.CONFIG_VERSION.CONFIG]
[SYSTEM.CRSD.TYPES.ora!database!type.DB_UNIQUE_NAME]
[SYSTEM.CRSD.TYPES.ora!database!type.FAILURE_THRESHOLD]
[SYSTEM.CRSD.TYPES.ora!database!type.INSTANCE_FAILOVER]
[SYSTEM.CRSD.TYPES.ora!database!type.ORACLE_HOME]
[SYSTEM.CRSD.TYPES.ora!database!type.RESTART_ATTEMPTS]
[SYSTEM.CRSD.TYPES.ora!database!type.ROLE]
[SYSTEM.CRSD.TYPES.ora!database!type.TYPE_ACL.CONFIG]
[SYSTEM.CRSD.TYPES.ora!database!type.USR_ORA_DB_NAME.CONFIG]
[SYSTEM.CRSD.RESOURCES.ora!net1!network]
[SYSTEM.CRSD.RESOURCES.ora!net1!network.INTERNAL]
[SYSTEM.CRSD.RESOURCES.ora!rac1!vip]
[SYSTEM.CRSD.RESOURCES.ora!gsd]
[SYSTEM.CRSD.RESOURCES.ora!gsd.INTERNAL]
[SYSTEM.CRSD.RESOURCES.ora!ons.CONFIG]
[SYSTEM.CRSD.RESOURCES.ora!asm]
[SYSTEM.CRSD.RESOURCES.ora!DATA!dg]
[SYSTEM.CRSD.RESOURCES.ora!scan2!vip]
[SYSTEM.CRSD.RESOURCES.ora!LISTENER_SCAN1!lsnr.INTERNAL]
[SYSTEM.CRSD.RESOURCES.ora!LISTENER_SCAN2!lsnr]
[SYSTEM.CRSD.RESOURCES.ora!LISTENER_SCAN3!lsnr.CONFIG]
[SYSTEM.CRSD.RESOURCES.ora!oc4j]
[SYSTEM.CRSD.RESOURCES.ora!rac2!vip.INTERNAL]
[SYSTEM.CRSD.RESOURCES.ora!LISTENER!lsnr.INTERNAL]
[SYSTEM.CRSD.RESOURCES.ora!orcl!db.INTERNAL]
[SYSTEM.OCR.MANUALBACKUP.ITEMS.0]
[SYSTEM.OCR.MANUALBACKUP.ITEMS.0.FILENAME]
[SYSTEM.OCR.MANUALBACKUP.ITEMS.0.LOC]
[SYSTEM.OCR.MANUALBACKUP.ITEMS.0.NODENAME]
[SYSTEM.OCR.MANUALBACKUP.ITEMS.0.TIMESTAMP]
[DATABASE.NODEAPPS.rac2]
[DATABASE.VIP_RANGE]
[DATABASE.LOG]
[DATABASE.ASM]
[DATABASE.ASM.rac1]
[DATABASE.ASM.rac1.+asm1]
[DATABASE.ASM.rac1.+asm1.ORACLE_HOME]
[DATABASE.ASM.rac1.+asm1.ENABLED]
[DATABASE.ASM.rac1.+asm1.VERSION]
[DATABASE.ASM.rac2]
[DATABASE.ASM.rac2.+asm2]
[DATABASE.ASM.rac2.+asm2.ORACLE_HOME]
[DATABASE.ASM.rac2.+asm2.ENABLED]
[DATABASE.DATABASES]
[CRS]
[CRS.CUR]
[CRS.HIS]
[CRS.SEC]
[CRS.STAGE]
[CRS.STAGE.node1]

The OCR dump has 3 main parts:

SYSTEM – Describes attributes of the clusterware such as interfaces, cluster name, cluster parameters such as misscount, locations of voting disks, location of OCR backups, cluster node names and numbers, hostnames etc

DATABASE – Describes metadata for all the resources managed by the clusterware.

CRS – Describes metadata for any Non-RAC resources that are managed for high availability by the clusterware.

Above highlighted information and the part of dump is to just give you an idea about how the information is stored in OCR. In reality OCR uses a file based repository to store configuration information in a series of key-value pairs using a directory tree like structure. So now after scanning the whole dump I can say that the OCR contains information about the resources controlled by Oracle Clusterware, including the following:

RAC databases and instances
SCAN listeners and local listeners
SCAN VIPs and local VIPs
Nodes and node apps types
ASM disk groups, volumes, file systems, and instances
OCRs Automatic and Manual backups information

Now that we have understood what is inside OCR the next natural curiosity is who maintains OCR who updates all this information in OCR, the answer is CRSd Process (and CSSd only during cluster startup), so even if you modify OCR via SRVCTL, CRSCTL, EM etc they communicate via CRSd. To explain this we need to know that Oracle uses a distributed shared cache architecture during cluster management to optimize queries against the cluster repository i.e. each node maintains a copy of the OCR in memory. Only one CRSd process (designated as the master) in the cluster performs any disk read/write activity. This master CRSd process refreshes the OCR cache on all cluster nodes. Clients (SRVCTL, CRSCTL etc) communicate with the local CRSd process to access the local copy of the OCR and contact the master CRSd process via the local CRSd process for any updates on the physical OCR binary file.

Because of this concept it is said that one should not modify OCR while any node is down ( as CRSd is down for that node ), so if need arises that one has to change the configuration in OCR when any node is down then a manual repair on the stopped node should be performed.

OCR and voting disks can be stored in Oracle ASM because the ASM’s partnership and status table is replicated on multiple disks and is extended to store OCR, the OCR can tolerate the loss of the same number of disks as are in the underlying disk group and can be relocated in response to disk failures.

There is much more to know about OCR but for today I guess this is enough to digest, would continue writing the post in OCR Internals part 2, I hope the above information helps you in understanding OCR, its purpose, its content, its usage and why was it required. Please comment below if you need any more information as in the complete dump of OCR, the description of all the components & keys, how CRSd manages and maintains OCR cache on all the nodes, how is the content updated when you alter any Clusterware configuration of a node, what happens when OCR is lost or corrupted