ORACLE RAC

Definition Oracle Real Application Cluster (RAC) is a cluster system at the application level. It uses shared disk architecture that provides scalability for all kind of applications. Applications without any modifications can use the RAC database. (Somewhat)‏

ORACLE RAC Goal Since the requests in a RAC cluster are spread evenly across the RAC instances, and since all instances access the same shared storage, addition of server(s) require no architecture changes etc. And a failure of a single RAC node results only in the loss of scalability and not in the loss of data since a single database image is utilized.

ORACLE RAC A requirement for Oracle Database 10g RAC cluster is a set of servers with shared disk access and interconnect connectivity. Since each instance in a RAC system must have access to the same database files, a shared storage is required that can be accessed from all RAC nodes concurrently. Shared Disks

ORACLE RAC The shared storage space can be used as raw devices, or by using a cluster file system or ASM. This article will address Oracle's Cluster File System OCFS and ASM. Note that Oracle 10g RAC provides it's own locking mechanisms and therefore it does not rely on other cluster software or on the operating system for handling locks. Types of Shared Storage

ORACLE RAC It is recommended to use newer Red Hat Enterprise Linux kernels since newer kernels might fix known database performance problems and other issues. However, you also need to make sure that the OCFS and ASM drivers are compatible with the kernel version! Upgrading the Kernel or Parameters

ORACLE RAC Each RAC node should have at least one static IP address for the public network and one static IP address for the private cluster interconnect. Configuring the Network

ORACLE RAC The private networks are critical components of a RAC cluster. The private networks should only be used by Oracle to carry Cluster Manager and Cache Fusion inter-node connection. A RAC database does not require a separate private network, but using a public network can degrade database performance (high latency, low bandwidth). Therefore the private network should have high-speed NICs (preferably one gigabit or more) and it should only be used by Oracle. Private Networks

ORACLE RAC Here is an example how the /etc/hosts file could look like: # Public hostnames for e.g. eth0 interfaces (public network)‏ rac1pub.puschitz.com rac1pub # RAC node rac2pub.puschitz.com rac2pub # RAC node rac3pub.puschitz.com rac3pub # RAC node 3 # Private hostnames, private network for e.g. eth1 interfaces rac1prv.puschitz.com rac1prv # RAC node rac2prv.puschitz.com rac2prv # RAC node rac3prv.puschitz.com rac3prv # RAC node 3 Network Example

ORACLE RAC Configuring Shared Storage fdisk -l Disk /dev/sda: 255 heads, 63 sectors, cylinders Units = cylinders of * 512 bytes Device Boot Start End Blocks Id System /dev/sda Linux /dev/sda Linux

ORACLE RAC If you use OCFS, it is important that the UID of "oracle" and GID of "oinstall" are the same across all RAC nodes. Otherwise the Oracle files on the OCFS filesystems on some nodes could either be "unowned", or they could even be owned by another user account. In my setup the UID and GID of oracle:dba is 700:700. su - root groupadd -g 700 dba # group of users to be granted with SYSDBA system privilege groupadd -g 701 oinstall # group owner of Oracle files useradd -c "Oracle software owner" -u 700 -g oinstall -G dba oracle passwd oracle Creating Oracle User

ORACLE RAC Since the Oracle Universal Installer (OUI) " runInstaller " is executed from the oracle account, some environment variables must be configured for the oracle account before OUI is started. If you use bash which is the default shell on Red Hat Linux (to verify your shell run: echo $SHELL ), execute the following commands: # Oracle Environment export ORACLE_BASE=/u01/app/oracle export ORACLE_SID=orcl1 # Each RAC node must have a unique Oracle SID! E.g. orcl1, orcl2,... export LD_LIBRARY_PATH=$ORACLE_HOME/lib Setting Oracle Accounts

ORACLE RAC At the time of this writing, OCFS only supports Oracle Datafiles and a few other files. Therefore OCFS should not be used for Shared Oracle Home installs. See Installing and Configuring Oracle Cluster File Systems (OCFS) for more information.Installing and Configuring Oracle Cluster File Systems (OCFS) Cluster File System

ORACLE RAC If you want to comply with Oracle's Optimal Flexible Architecture (OFA), then you don't want to place the database files in the /u01 directory but in another directory like /u02. This is not a requirement but if you want to comply with OFA, then you might want to create the following directories as well: su - root mkdir -p /u02/oradata/orcl chown -R oracle.oinstall /u02 Creating Oracle Directories

ORACLE RAC Note that it is important for the Redo Log files to be on the shared disks as well. To partition the disks, you can use the fdisk utility: su - root fdisk Here is an example how to create a new 50 GB partition on drive /dev/sda : su - root # fdisk /dev/sda Creating Partitions

ORACLE RAC The Oracle Cluster File System (OCFS) was developed by Oracle to overcome the limits of Raw Devices and Partitions. It also eases administration of database files because it looks and feels just like a regular file system. OCFS only supports Oracle Datafiles and a few other files: - Redo Log files - Archive log files - Control files - Database datafiles - Shared quorum disk file for the cluster manager - Shared init file (srv)‏Oracle Cluster File System (OCFS) Installing and Configuring OCFS

ORACLE RAC To create the OCFS filesystems, you can use the ocfstool : su - root ocfstool - Select "Task" - Select "Format" Alternatively, you can execute the " mkfs.ocfs " command to create the OCFS filesystems: su - root mkfs.ocfs -F -b 128 -L /u02/oradata/orcl -m /u02/oradata/orcl \ -u `id -u oracle` -g `id -g oracle` -p 0775 Creating OCS File Systems

ORACLE RAC Add lines to the /etc/fstab file similar to the following example: /dev/sda1 /u02/oradata/orcl ocfs _netdev 0 0 The " _netdev " option prevents the OCFS filesystem from being mounted until the network has first been enabled on the system. To make sure the ocfs.o kernel module is loaded and the OCFS file systems are mounted during the boot process, enter: su - root # chkconfig --list ocfs ocfs 0:off 1:off 2:off 3:on 4:on 5:on 6:off Having File System a Boot Time

ORACLE RAC Oracle uses the Linux kernel module hangcheck-timer to monitor the system health of the cluster and to reset a RAC node in case of failures. The hangcheck-timer module uses a kernel-based timer to periodically check the system task scheduler. This timer resets the node when the system hangs or pauses. This module uses the Time Stamp Counter (TSC) CPU register which is a counter that is incremented at each clock signal. The hangcheck-timer module comes now with the kernel: find /lib/modules -name "hangcheck-timer.o" Configue Hangcheck Timer in Kernel

ORACLE RAC When you run the Oracle Installer on a RAC node, it will use ssh to copy Oracle software and data to other RAC nodes. Therefore, the oracle user on the RAC node where Oracle Installer is launched must be able to login to other RAC nodes without having to provide a password or passphrase. The following procedure shows how ssh can be configured that no password is requested for oracle ssh logins. To create an authentication key for oracle, enter the following command on all RAC node: Remote Access for RAC Nodes

ORACLE RAC Sizing Swap Space Setting Shared Memory Checking /tmp Space Setting Semaphores Setting File Handles More System Setup

ORACLE RAC Cluster Ready Services (CRS) contains cluster and database configuration information for RAC, and it provides many system management features. CRS accepts registration of Oracle instances to the cluster and it sends ping messages to other RAC nodes. If the heartbeat fails, CRS will use shared disk to distinguish between a node failure and a network failure. Once CRS is running on all RAC nodes, OUI will automatically recognice all nodes on the cluster. This means that you can run OUI on one RAC node to install the Oracle software on all other RAC nodes. Installing Cluster Ready Services

ORACLE RAC Make sure that the oracle user can ssh to all RAC nodes without ssh asking for a passphrase. This is very important because otherwise OUI won't be able to install the Oracle software on other RAC nodes. The following example shows how ssh-agent can do the authentication for you when the oracle account logs in to other RAC nodes using ssh. Open a new terminal for the RAC node where you will execute runInstaller and use this terminal to login from your desktop using the following command: $ ssh -X Starting The Installer

ORACLE RAC Now configure ssh-agent to handle the authentication for the oracle account: oracle$ ssh-agent $SHELL oracle$ ssh-add Enter passphrase for /home/oracle/.ssh/id_dsa: Enter your passphrase Identity added: /home/oracle/.ssh/id_dsa (/home/oracle/.ssh/id_dsa)‏ oracle$ Configure ssh-agent

ORACLE RAC In order to install the Cluster Ready Services (CRS) R1 ( ) on all RAC nodes, OUI has to be launched on only one RAC node. To install CRS, insert the "Cluster Ready Services (CRS) R1 ( )" CD (downloadedd image name: " ship.crs.cpio.gz "), and mount it on e.g. rac1pub: su - root mount /mnt/cdrom Use the oracle terminal that you prepared for ssh at Automating Authentication for oracle ssh Logins and execute runInstaller :Automating Authentication for oracle ssh Logins oracle$ /mnt/cdrom/runInstaller Install Cluster Ready Services

ORACLE RAC The following procedure shows the installation of the software for Oracle Database 10g Software R1 ( ) with Real Application Clusters (RAC). Note that Oracle Database 10g R1 (10.1) OUI will not be able to discover disks that are marked as Linux ASMLib. Therefore it is recommended to complete the software installation and then to use dbca to create the database, see for more information. Installing ORACLE with RAC

ORACLE RAC The following steps show how to use dbca to create the database and its instances. Oracle recommends to use dbca to create the RAC database since the preconfigured databases are optimized for ASM, server parameter file, and automatic undo management. dbca also makes it much more easier to create new ASM disk groups etc. Setting Up the Database with dbca

ORACLE RAC Database File Locations: Select "Use Oracle-Managed Files" Database Area: +ORCL_DATA1 Click Next I selected TAF Policy "Basic". Click Next - Initialization Parameters: Change settings as needed. Click Next - Database Storage: Change settings as needed. Click Next - Creation Options: Check "Create Database" Click Finish Now the database is being created. DBCA concluded (creating database)‏

ORACLE RAC RAC requires a CFS RAC is complex RAC provides transactions (OLTP)‏ Summary