1. Data Dependent Routing may not be necessary when using Oracle RAC Ken Gottry Apr-2003 Through Technology Improvements in: Oracle 9i - RAC Oracle 9i - CacheFusion Solaris - RSM Sun Cluster – SunFire Link

2. www.gottry.com 2 Objective To provide a brief overview of several new technologies that have been implemented by Oracle and Sun over the past 18 months. These include: Oracle 9i RAC database cluster Oracle 9i CacheFusion Solaris Remote Shared Memory (RSM) Sun Cluster SunFire Link To suggest that, based on the above improvements, application logic to implement data dependent routing may no longer be as important when using an Oracle RAC database cluster.

3. www.gottry.com 3 Agenda Executive Summary HA-Oracle vs. OPS/RAC Pinging in OPS Pinging in RAC Data dependent routing (DDR) Oracle 9i CacheFusion Solaris remote shared memory (RSM) Sun Cluster Interconnect – SunFire Link

4. www.gottry.com 4 Executive Summary What was called Oracle Parallel Server (OPS) in 8i is now called Real Application Cluster (RAC) in 9i CacheFusion in 9i reduces pinging degradation from 20% in OPS to 5-10% Oracle 9i can use Solaris Remote Shared Memory (RSM) to move CacheFusion into the kernel level. Pinging degradation may be reduced to 3-5% Sun Cluster supports SunFire Link, a 1.6 Gbps pipe between cluster nodes with less than 1 ms latency. Up to 6 SunFire Link interconnects between nodes will allow striping of data transfer. Pinging degradation may be reduced to 1-3% With such reduction in pinging degradation, is data dependent routing (DDR) a design concern any more?

5. www.gottry.com 5 HA-Oracle vs. RAC HA-Oracle Only one DB server active at a time Failover may take a long time RAC Both DB servers active so throughput is often 80-90% more than with HA- Oracle Distributed Lock Mgr (DLM) called Global Cache Service (GCS) in 9i Failover is immediate Requires application coding Failover

6. www.gottry.com 6 Pinging with 8i OPS Pinging Reduced throughput when DB node #1 has to ask DB node #2 if it has the needed block before DB node #1 can update it Oracle 8i OPS DB node #2 had to flush the block to disk before DB node #1 could have it Throughput was degraded about 20% with OPS pinging. Example: assume one DB node could process 100 tps. When adding a second DB node, you would expect the OPS database cluster to process 200 tps. However, due to the pinging overhead, you would normally see (100 + 100) – (20% * (100 + 100)) = 200 – 40 = 160 tps

7. www.gottry.com 7 Pinging with 9i RAC Oracle 9i RAC Using CacheFusion, DB node #2 pushes the block to DB node #1 over the cluster interconnect. Pinging still occurs within RAC, but is much faster because the block is transferred between cache without a disk write by DB node #2 Throughput degraded about 10% with RAC pinging. Example: assume one DB node could process 100 tps. When adding a second DB node, you would expect the RAC database cluster to process 200 tps. However, due to the pinging overhead, you would normally see (100 + 100) – (10% * (100 + 100)) = 200 – 20 = 180 tps

8. www.gottry.com 8 Data Dependent Routing (DDR) App knows that DB server #1 is the primary handler of the portion of the DB containing Patient ID’s 1- 500. So, app sends SQL request for patient ID 200 to DB server #1 to minimize impact of pinging App knows that DB server #2 is the primary handler of the portion of the DB containing Patient ID’s 501- 1000. So, app sends SQL request for patient ID 800 to DB server #2 to minimize impact of pinging Notice ping still happens, but no block transfer is required. It’s the block transfer that can degrade throughput by up to 5-20% To minimize the impact of pinging, architects often partition the DB, making one DB node primarily responsible for one-half the DB and the other DB node primarily responsible for the other half. The application must then contain data dependent routing logic that decides to which DB node to send each SQL call

9. www.gottry.com 9 CacheFusion and Remote Shared Memory (RSM) Oracle 9i CacheFusion makes the cache on multiple DB nodes act as one. This speeds up block transfer when it’s needed. Taking a closer look, CacheFusion is implemented at the application (Oracle) level Solaris’ Remote Shared Memory (RSM) allows clustered apps to share memory at the kernel level. Oracle 9.1 implements RSM-API

10. www.gottry.com 10 SunFire Link Interconnect Nodes of a cluster use a private network connection between the nodes to communicate. Heartbeat (“are you alive”) info is exchanged over the cluster interconnect Previously Sun Cluster supported two types of interconnect: ethernet (100Mbps) proprietary SCI (200 Mbps) In Apr-2003, Sun Cluster announced support for proprietary SunFire Link interconnect (1.6Gbps). Up to 6 SFL interconnects can be used to stripe the data as it is transferred

11. www.gottry.com 11 Is Data Dependent Routing Needed? This chart and table show the relative improvement in throughput using the new technologies. Perhaps this improvement is good enough to avoid adding data dependent routing logic to your application. Based on a 2 node DB cluster with each node capable of 100 tps ConfigurationDegradationThroughputTotal OPS20%80 + 80160 RAC with CacheFusion10%90 + 90180 RAC with RSM7%93 + 93186 RAC with RSM and SunFire Link3%97 + 97194 Ideal0%100 + 100200