1. Managing Oracle Enterprise Manager Cloud Control 12c with Oracle Clusterware Session ID#UGF9794 Leighton L. Nelson Mercy

2. About Me Oracle DBA for 10+ years Oracle ACE Oracle Certified Expert: RAC and Grid Infrastructure 11gR2 Co-Author Expert Oracle Enterprise Manager Cloud Control 12c – Apress 2013 Oracle RAC SIG US Events Chair/IOUG Liaison

3. Agenda OEM 12c Architecture Overview OEM High Availability Level 2 HA Configuration Oracle Clusterware Setup OEM Installation OEM Failover Management Repository Configuration

4. OEM 12c Architecture Overview

5. OEM High Availability Levels LevelDescriptionLoad Balancer RequiredCost 1OMS and Repository on separate hosts. No redundancy. No$ 2OMS installed on shared storage with VIP used in active/passive failover. Repository Database using local Data Guard. No$$ 3Multiple OMSs in active/active configuration. Repository using RAC Database with local Data Guard Yes, At primary site$$$ 4Primary OMS in active/active configuration with RAC Repository database. Standby OMS at DR site in active/active configuration. Standby RAC database at DR site Yes. At pimary and standby sites $$$$

6. Level 2 – Active/Passive HA A minimum of 2 Servers Required OMS binaries installed on shared filesystem NFS/OCFS2/DBFS/ACFS OMS can run on one node in cluster at any given time Data Guard for Management Repository

7. Level 2 – Active/Passive OMS

8. Setup virtual hostname and IP address (VIP) Clusterware VIP Virtual hostname should resolve to unique IP Install OMS on shared disk Create Clusterware resource for OMS Failover

9. Oracle Clusterware Setup Clusterware can be used to create/manage VIP 11.2+ uses appvipcfg for VIP /bin/appvipcfg create -network=1 \ -ip=192.168.1.0 \ -vipname=omsvip \ -user=root VIP can be created on non-default network In Oracle 12c Flex Clusters app vips can be created on leaf nodes Allow Oracle Grid Infrastructure software owner (e.g. grid) to run the script to start the VIP. /bin/crsctl setperm resource omsvip -u user:grid:r-x

10. Oracle Clusterware Setup Start the VIP as the GI owner e.g. grid /bin/crsctl start resource omsvip Check the status of the VIP /bin/crsctl status resource omsvip The status of the output should be similar to the following: NAME=omsvip TYPE=app.appvip_net1.type TARGET=ONLINE STATE=ONLINE on oms1

11. Oracle Clusterware Setup Check if virtual hostname and VIP are resolvable nslookup Also do a reverse lookup of the IP address. nslookup Verify that the IP address returned from the nslookup output is running on the OMS host. ifconfig –a | grep

12. OEM Installation Create ORACLE_HOME for the OMS on the shared storage on all nodes in the cluster mkdir –p /u01/app/oms_share Create Oracle Inventory directory under ORACLE_HOME for the OMS on all nodes mkdir /u01/app/oms_share/oraInventory Create the inventory pointer in the oraInventory directory vi oraInst.loc inventory loc=/u01/app/oracle/oms_share/oraInventory inst_group=oinstall

13. OEM Installation Install OEM by setting ORACLE_HOSTNAME environment variable runInstaller -invPtrloc /u01/app/oms_share/oraInst.loc ORACLE_HOSTNAME=omsvip.example.com –debug

14. OEM Failover To manually relocate the VIP to another host in the cluster issue the following command. $crsctl relocate res omsvip CRS-2673: Attempting to stop 'omsvip' on 'oms1' CRS-2677: Stop of 'omsvip' on 'oms1' succeeded CRS-2672: Attempting to start 'omsvip' on 'oms2' CRS-2676: Start of 'omsvip' on 'oms2' succeeded Check if the IP address associated with the VIP is running on the relocated host. ifconfig –a|grep

15. OEM Failover Establish IP on failover server (done through Clusterware) Start listener (if part of same failover group) Start database (if required) Set the ORACLE_HOSTNAME environment variable to the virtual hostname. Continuing with our example we use the command below. export ORACLE_HOSTNAME=omsvip.example.com Start the OMS on the new node $OMS_HOME/bin/emctl start oms

16. Add OEM Clusterware resource OEM can be added as Clusterware resource Administrator managed (static, 2 – nodes) Policy managed (dynamic, > 2 nodes) Handled through Agent Framework C/C++ Create Action Script Store on shared storage Specify START/STOP/CHECK/ABORT routines

17. Example $ crsctl add server pool oem_sp -attr "PARENT_POOLS=Generic, SERVER_NAMES=oms1 oms2” $ crsctl add resource oem -type cluster_resource -attr "ACTION_SCRIPT=/opt/cluster/scripts/oem12c.scr, PLACEMENT='restricted', SERVER_POOLS=oem_sp, CHECK_INTERVAL='30', RESTART_ATTEMPTS='2', START_DEPENDENCIES='hard(omsvip)', STOP_DEPENDENCIES='hard(omssvip)'"

18. Clusterware resource Oracle 12c introduced a new generic_application resource type crsctl add resource oem -type generic_application -attr "START_PROGRAM='$OMS_HOME/bin/emctl start oms', STOP_PROGRAM='$OMS_HOME/bin/emctl stop oms', CLEAN_PROGRAM='$OMS_HOME/bin/emctl stop oms –all -force', CHECK_PROGRAM='$OMS_HOME/bin/emctl status oms'

19. Management Repository Can be configured using Data Guard with Fast-Failover Also can be configured with RAC/RAC One Node Storage becomes SPOF Listeners should be reachable from all nodes in cluster Database should be started before starting OEM

20. Configure OEM with Data Guard Repo Create database service for fast failover.

21. Configure OEM with Data Guard Repo emctl config oms -store_repos_details -repos_conndesc '"(DESCRIPTION=(FAILOVER=ON)(ADDRESS_LIST=(A DDRESS=(PROTOCOL=TCP)(HOST=repo1)(PORT=15 21))(ADDRESS=(PROTOCOL=TCP)(HOST=repo2)(POR T=1521)))(CONNECT_DATA=(SERVICE_NAME=emrep) )(FAILOVER_MODE=(TYPE=select)(METHOD=basic)))"' -repos_user sysman

22. Configure OEM with RAC/RAC One Node Repo emctl config oms -store_repos_details -repos_conndesc " (DESCRIPTION= (ADDRESS=(PROTOCOL=TCP) (HOST=emrep-scan.example.com)(PORT=1521) ) (CONNECT_DATA=(SERVER=DEDICATED) (SERVICE_NAME = emrep)))" -repos_user sysman

23. References How to Configure Grid Control OMS in Active/Passive CFC Environments failover / HA (Doc ID 405642.1) How to Configure the OMS Connect String when Repository is in a Dataguard setup [ID 1328768.1] Oracle® Clusterware Administration and Deployment Guide12c Release 1

24. COLLABORATE14.IOUG.ORG Network with 5,000+ database and application professionals 5 days of in-depth education built by users, for users Complimentary Pre-Conference Workshops for IOUG registrants April 7 – 11, 2014 The Venetian Resort, Las Vegas, NV Attend for free! IOUG’s Call for Speakers is now open collaborate14.ioug.org/call-for-speakers

25. Communities Training Close to Home Maximum Availability Architecture with Oracle’s Larry Carpenter and Joe Meeks October 15: Milwaukee, WI October 16: Rochester, NY Performance and Internals with Craig Shallahamer November 6: Chicago, IL November 8: Atlanta, GA Get more information at www.ioug.org/masterclass or visit the IOUG kiosk in Moscone West, 2 nd floor IOUG Master Class Series Returns Coming soon to a city near you! One Day Interactive Training Events to meet your educational needs

26. Come See IOUG at the User Group Pavilion Win a Free Registration to COLLABORATE 14! Stop by the IOUG kiosk in the User Group Pavilion in Moscone West, 2nd floor, to pick up a lucky poker chip and see if you are a winner of a free registration to COLLABORATE 14 in Las Vegas!

27. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 27 Complimentary eBook Register Now www.mhprofessional.com/dbsec Use Code: db12c

28. THANK YOU http://blogs.griddba.com linkedin.com/pub/leighton-nelson/4/6b8/a7a @leight0nn leighton.nelson@mercy.net