1. Clustering  Types of Clustering

2. Objectives At the end of this module the student will understand the following tasks and concepts. What clustering is and why you would want it Clustering options Differences between various types of clustering; advantages and disadvantages Factors to consider when choosing a cluster type

3. What is a cluster? Multiple systems performing a single function Multiple systems performing a single function Black box Black box A cluster is a group of independent servers that function as a single system. OR

4. Q: What is RAC? A: Oracle ’ s clustered database for real (i.e. everyday) applications Oracle Oracle RAC Clustered it becomes … Database Instance Database Instances

5. What Is a RAC Cluster? Nodes Interconnect Shared disk subsystem Instances Database Node Interconnect Disks

6. What is Real Application Clusters? Two or more interconnected, but independent servers One instance per node Multiple instances accessing the same database Database files stored on disks physically or logically connected to each node, so that every instance can read from or write to them

7. Database vs Instance RAC Cluster consists of …. One or more instances One Database residing on shared storage Node 1 Instance 1 Node 2 Instance 2 Interconnect Shared Storage Local Disk Database

8. What RAC provides  Increased Availability – A server failure does not interrupt surviving nodes  Increased Scalability – Scale outside the box  Increased Flexibility – Grow as your demand grows  Reduced Cost – No idle servers – better server utilization – Leverage cost savings of smaller servers  Lower initial cost  Lower support cost – Consolidation

9. Database Cluster Types  Shared Everything – Shared Disk/Cache – Oracle and IBM Mainframes – More Reliable As You Add Computers – No Data Partitioning Required  Shared Nothing – Private Disk/Cache – Microsoft and IBM Unix/NT – Less Reliable As You Add Computers – Static Data Partitioning DataA-Z DataA-EDataF-KDataL-SDataT-Z

10. Server 1 Server 2 Server N ClusterWareClusterWare ClusterWare Key Software Components Server 3 ClusterWare OSOSOSOS Raw / CFS / NAS / ASM Oracle RAC Listener Oracle RAC Listener Oracle RAC Listener Oracle RAC Listener

11. Cluster Definitions Shared Nothing (Federated) Replicated Site Shared Disk Failover Active/Passive Active/Passive Active/Active Active/Active Shared Everything

12. Shared Nothing Cluster Only one CPU is connected to a disk May have shared memory MPP ( Systems are Shared Nothing MPP (Massively Parallel Systems) Systems are Shared Nothing Other vendors have “Shared Nothing” clusters

13. Federated (Shared Nothing) Cluster Distributed database (separate database on each machine) Data is spread across nodes; each machine has part of the data Function is spread across nodes Two-Phase Commit Database Server Got it? Got it! Good! 1. 2. 3.

14. Replicated System Data replicated at the server (network) level or at the storage (SAN) level Multiple copies of the same database Most common implementation is Active/Passive Failover between nodes Database Server Server level Replication Storage level Replication Active Node Passive Node or

15. Shared Disk Cluster Shared file system Multiple systems attached to the same disk All nodes must have access to data Only one database instance; only one node has “ownership” of the shared disk Synchronization between systems; If one node fails, then the other takes over

16. Failover Cluster One system is a standby system for another Only one system doing work at a time Pseudo-Shared Disk Limited scalability in active/passive mode

17. Failover Clustering Fault tolerant systems; highly available Basic failover clusters don’t scale beyond two nodes Users Database Server

18. Active/Passive vs. Active/Active Both are failover only Active/Passive One node is active One node is active The other is passive until failover The other is passive until failoverActive/Active Still uses active/passive technology Still uses active/passive technology 2 separate databases 2 separate databases One is active on node A and passive on node B One is active on node A and passive on node B The second database is active on node B and passive on node A. The second database is active on node B and passive on node A. Separate applications and user connections to each of the different databases Separate applications and user connections to each of the different databases

19. Active/Passive Node A is active Node A is active Node B is passive until/unless Node A fails Node B is passive until/unless Node A fails Only one Oracle license is required Node ANode B

20. Active/Passive Node ANode B X If Node A fails …

21. Active/Passive Node B becomes active Node B becomes active Node A is dead (definitely passive!) until repaired and then “failed back” if necessary. Node A is dead (definitely passive!) until repaired and then “failed back” if necessary. Node ANode B X

22. Active/Active Application Group A and User Group A are active on Node A Application Group B and User Group B are active on Node B Each node serves as failover for the other. 2 separate databases. Both nodes are not accessing the same data at the same time. Oracle license required on each node Node A Application A User Group A Passive Fail- over for B Node B Passive Fail- over for A Application B User Group B

23. Storage Options  Raw Files – All platforms  Clustered File System – Windows & Linux - OCFS – Solaris & AIX – Veritas CFS  Automatic Storage Management (ASM) – All platforms  NAS (via NFS) – Linux & Solaris

24. Real Applications Clusters  Use RAC for scalability – Add instances against same database files providing more Oracle processes and increasing number of users – Add or remove nodes when needed  Oracle 10g provides platform independent Cluster Ready Services (CRS) to handle failover of services to surviving nodes

25. RAC Instance Architecture Public Network Node 1 ASM inst DB inst 1 CRS ASM inst DB inst 2 CRS ASM inst DB inst 3 CRS Node 2 Node 3 OCR Vote DB Nodeapps: Vip, ons, gsd private network private Nodeapps: Vip, ons, gsd

26. OCR And Voting Disk Oracle Cluster Registry: The OCR maintains cluster configuration information that is used by each node of the cluster to determine the state of the cluster. OCR also maintains information about cluster resources: Databases Instance Services Each node in the cluster maintains a copy of the OCR in memory for better performance and also responsible for updating the OCR as required in shared storage. Voting Disk: The Voting Disk Files are used by Oracle Clusterware to determine which nodes are currently members of the cluster.

27. ASM Disk Groups, Disks and Database Files Disk Group 1 Disk 1Disk 4 Disk Group 2 Disk 2Disk 5Disk 6Disk 7 Disk 3 Disk Group 3 File 3 File 4 File 5 File 2 File 5 File 6 File 1

28. RAC Scalability & Availability  High Availability – Node & Database instance no longer represent single point of failure – survive node and instance failures

29. RAC Network Checklist Public adapter should be first Private adapter should be second Ping node’s public hostname to verify Ping each node’s public & private hostname Don’t use the name ‘Private’ for the Private Network Network Connections Advanced-> Advanced Setting

30. RAC Best Practices  Eliminate Single Points of Failure – NIC’s, Switches, Interconnect, Shared Storage, Power Supplies – Understand cost vs. availability tradeoff  Use fastest switch available for private interconnect – Disable additional protocols such as spanning tree protocol – Increase MTU size as high as switch allows – I.e. 9000 – Allow cards and switch ports to autonegotiate speed.  Use static IP addresses – Public LAN resolved by DNS and hosts file – For cluster interconnect use non-routable IP (10.X or 192.168.X)

31. Clustering Solutions from Oracle Oracle Failsafe Oracle Data Guard Advanced Replication Shared Nothing Cluster Oracle Parallel Server Real Application Clustering (RAC)

32. Oracle Fail Safe  Integrated with Microsoft Clustering, Fail Safe is a core feature included with every Oracle version  In the event of a system failure, Oracle Fail Safe works with Microsoft Cluster Server to restart Oracle databases and applications on a surviving cluster node  MSCS and Fail Safe uses “share-nothing” architecture (only one node can access shared datafiles at any time)

33. Failsafe MS Clustering Enabled MS Clustering Enabled Two servers one disk subsystem Switches in the event of a hardware failure Requires recovery

34. Oracle Fail Safe Fail Safe Server ResourceMSCS Resource DLL Fail Safe Server ResourceMSCS Resource DLL Node A Node B Fail Safe Manager Cluster Disks Private Disk

35. Before Failover… Primary Server for Virtual Server C Database Clients Secondary Server for Virtual Server C Private Disk(s) Database Disks Shared I/O Interconnect Group 1: Disk Resource IP Address Resource Network Name Resource Net Listener Resource Oracle Database Resource Virtual Server: C Node: A Oracle Fail Safe Database Instance Private Disk(s) Node: B Oracle Fail Safe Server MSCS Oracle Fail Safe Server MSCS

36. Node goes down… Primary Server for Virtual Server C Database Clients Secondary Server for Virtual Server C Private Disk(s) Database Disks Shared I/O Interconnect Group 1: Disk Resource IP Address Resource Network Name Resource Net Listener Resource Oracle Database Resource Virtual Server: C Node: A Oracle Fail Safe Database Instance Private Disk(s) Node: B Oracle Fail Safe Server MSCS Oracle Fail Safe Server MSCS

37. After failover… Primary Server for Virtual Server C Database Clients Secondary Server for Virtual Server C Private Disk(s) Database Disks Shared I/O Interconnect Group 1: Disk Resource IP Address Resource Network Name Resource Net Listener Resource Oracle Database Resource Virtual Server: C Node: A Oracle Fail Safe Database Instance Private Disk(s) Node: B Oracle Fail Safe Server MSCS Oracle Fail Safe Server MSCS

38. Fail Safe Manager

39. Oracle Data Guard  Data Guard is Oracle’s Disaster Recovery product which maintains and monitors one or more standby databases to protect enterprise data from failures, disasters, errors, and corruptions  Standby databases, which can be located across large geographic regions away from the primary database, can be switched to the production role if a problem occurs with the primary  Can use different Windows versions for primary and standby (2003 for primary, 2000 for standby)  DG is free with Enterprise Edition of RDBMS http://www.oracle.com/technology/deploy/availability/htdocs/DataGuard Overview.html

40. Oracle Data Guard Mirrored Server Physical Standby Archive Logs are applied to the remote database Archive Logs are applied to the remote database Switchover occurs in the event of a failure Switchover occurs in the event of a failure Logical Standby Log Miner technology is used to generate SQL Log Miner technology is used to generate SQL Standby Database can also be used for read-only reporting Standby Database can also be used for read-only reportingAdvantages Safe from user failure Safe from user failure Can be in different location Can be in different location No recovery required No recovery required

41. www.brianhitchcock.net Brian Hitchcock October 23, 2007Page 41 DataGuard Classic* Primary Database Online Redo Logs Archived Redo Logs Standby Database Archived Redo Logs DataGuard Standby mounted, recovering or Standby read-only, no apply Can switch back and forth -Primary becomes standby -Standby becomes primary *Before choice of physical or logical standby

42.  RAC + DataGuard – Eliminates physical location as SPOF Maximum Availability Architecture (MAA) RAC Cluster NYC Primary Single Instance Boston Standby Data Guard

43. Maximum Availability Architecture WAN Traffic Manager Dedicated Network Primary SiteSecondary Site Data Guard - Flashback - RMAN - Grid Control RAC 10g AS - Flashback - RMAN - Grid Control

44. Advanced Replication Uses Updatable-Snapshots Replicates to another system Systems stay in sync

45. Oracle Parallel Server Shared disk cluster product Loosely Coupled Scalable performance No downtime in the event of a system failure Replaced by RAC in 9i

46. Factors to Consider for Clustering Which do you need most? High Availability – Failover Clusters, Synchronous Replication, Data Guard High Availability – Failover Clusters, Synchronous Replication, Data Guard Performance scalability – Active/Active failover clusters, N-to-N failover clusters Performance scalability – Active/Active failover clusters, N-to-N failover clusters Both – Oracle RAC Both – Oracle RAC Administration complexity Failover clusters – relatively low Failover clusters – relatively low Oracle RAC – relatively high Oracle RAC – relatively high Substantially less complex for 10g RAC than 9i RAC Substantially less complex for 10g RAC than 9i RAC Local or long distance? Local – Failover, RAC Local – Failover, RAC Remote – Federated database, Replication, Standby database/Data Guard Remote – Federated database, Replication, Standby database/Data Guard Oracle license costs Active/Passive failover clusters – active nodes only Active/Passive failover clusters – active nodes only Active/Active failover clusters, RAC – per node Active/Active failover clusters, RAC – per node