Oracle 11g Real Application Clusters Advanced Administration Natik Ameen Session #352

Introduction Natik Ameen, Senior Oracle DBA DBA for 8 years, with 4 years on RAC 9i, 10g Financial, Gaming and Aviation industries Intended for everyone who has basic knowledge of RAC

Objective RAC overview ASM Nodeapps Metrics collection Wait events Best Practice 11g New features

RAC Overview Clustered nodes Cluster Interconnect Virtual IP Shared storage NodeApps

RAC Architecture Clustered nodes Interconnect Network Shared storage

SGA Structure & Processes Global Resource Directory : Maintains mapping of data in SGA Global Cache Server: Synchronizes Buffer cache across instances Global Enqueue Services: Maintains coherency in data dictionary & library caches Lock Manager Server: Transports blocks for Cache Fusion between nodes.

Cache Fusion What is it Concurrency & Consistency Scalability

Interconnect Network Main component for “Cache Fusion” IC uses private network User Datagram Protocol(UDP) Reliable Datagram Socket Protocol(RDS) > 10.2.0.3 Typical bandwidth utilization 20-30%, saturated > 70%

ASM Overview ASM Instance Disk Group Disk Files ASMLib

ASM Instance Scaled down version of DB instance DB instance access DB files

Disk Groups Striping Mirroring Failure groups

ASM Files Name pre-pended with ‘+’

ASMLib Kernel driver with efficient access to DB files

NodeApps Listener Oracle Notification Services Fast Application Notification Fast Connection Failover

Metrics Collection AWR Reports OS Utilities Bench Marking

AWR Report The RAC specific statistics include: Instance Global Cache Load Profile Global Cache Efficiency percentages Messaging Statistics Top % CR and Current Block Segments

Cache Fusion data block & messaging traffic Calculate network traffic

RAC Wait Events Most of these wait events occur during Cache Fusion GC cr/current block 2-way, 3-way GC cr/current block congested GC cr/current block busy GC current grant busy GC cr/current block request GC cr block lost

GC wait events Classified under “cluster wait lass”. Characterized as “current” or as “CR”. “current” classified as all blocks read into memory for the very first time CR denotes block for read access Master node maintains complete information on the resource, others maintain information on locally held locks

gc current block 2-way Instance A requests block from master instance B If the block is available on B then it is sent to A

gc current block 3-way Instance A requests block from master instance B B does not have block but directs instance holding block or directs request to disk Three hops maximum, not dependant on number of nodes

GC cr/current block congested Repeated requests by foreground processes, not serviced by LMS Indicates LMS not able to keep up Queue lengths & scheduling delays in OS, can cause LMS delays GC cr/current block busy Delay for some reason, before block sent to requestor Could be due to I/O saturation GC current grant busy Permission to access the block granted, but blocked by other requests ahead of it GC cr/current block request Wait time, when cr or current block is being retrieved GC block lost Lost blocks possibly due to faulty interconnect network

Where to get more information Other sessions here at IOUG-A Live! RAC Special Insterest Group(RAC SIG) Consulting companies (depending on specialization)

Questions? Lots of things to cover If we didn’t cover something you wanted to hear, please contact me

Oracle 11g Real Application Clusters Advanced Administration THANK YOU! Please fill out evaluations!