1. © 2005 Solution Beacon, LLC. All Rights Reserved. Release 11i Workshops Release 11i … Please Add RAC Session Leader Kevin Dahl, Solution Beacon Release 11i Workshops San Ramon, CA Worthington, MA Los Angeles, CA St. Louis, MO Orlando, FL www.solutionbeacon.com TRAIL to TEXAS sm

2. © 2005 Solution Beacon, LLC. All Rights Reserved. 2 Introduction The Challenge The Challenge – Customer wanted to run Release 11i and could only afford to be down 5 minutes The 50,000 Foot Solution The 50,000 Foot Solution – Implement SUN Cluster 3.0 on 2 nodes – Implement 11i with 9i RAC – Utilize 2 apps servers

3. © 2005 Solution Beacon, LLC. All Rights Reserved. 3 Introduction Install 11iInstall 9iUpgrade 8i to 9i Configure RAC Install SUN Cluster 3.0 OVERVIEW

4. © 2005 Solution Beacon, LLC. All Rights Reserved. 4 Choosing the Oracle Software 11i and RAC 11i and RAC – Customer decided Release 11i would best meet their software needs – Explained that Oracle9i Real Application Clusters (RAC) was an architecture that allowed multiple instances to access a shared database of datafiles – Customer was shown that Oracle9i RAC would meet their 5 minutes down time requirement

5. © 2005 Solution Beacon, LLC. All Rights Reserved. 5 Choosing the Hardware What was chosen? What was chosen? – SUN V880s. 2 clustered nodes for backend, and 2 non-clustered nodes for middle tier – A T3 storage array is in the same storage enclosure as the SunFire V880. That means there are two T3 storage arrays connected via an optical hub. The arrays provide both storage that is local to a given database server and storage that is shared between the two database servers

6. © 2005 Solution Beacon, LLC. All Rights Reserved. 6 Implementation – Hardware

7. © 2005 Solution Beacon, LLC. All Rights Reserved. 7 Implementation – SUN Cluster 3.0 Install SUN Cluster 3.0 (SUN did this) Install SUN Cluster 3.0 (SUN did this) – A cluster is a collection of loosely coupled computing nodes that provides a single client view of network services or applications, including databases, web services, and file services – Each cluster node is a standalone server that runs its own processes. These processes can communicate with one another to form a virtual single system

8. © 2005 Solution Beacon, LLC. All Rights Reserved. 8 11i RAC Leaving the Overview – The Detail

9. © 2005 Solution Beacon, LLC. All Rights Reserved. 9 Implementation – Release 11i Install Release 11i via Rapidwiz Install Release 11i via Rapidwiz – 2-tier – post install steps – backed up database

10. © 2005 Solution Beacon, LLC. All Rights Reserved. 10 Get 11i Ready for 9i Applied Interoperability Patch Applied Interoperability Patch – This patch is necessary for Oracle9i to be used in place of Oracle8i when using Release 11i – The patch number is 1833692

11. © 2005 Solution Beacon, LLC. All Rights Reserved. 11 Install Oracle9i (9.0.1) runInstaller runInstaller – chose enterprise installation type – did NOT install a database Modify init.ora and iflecbo.ora Modify init.ora and iflecbo.ora – copied init and ifilecbo and modified as follows:  remove obsolete parms  set aq_tm_processes = 0  set job_queue_processes = 0

12. © 2005 Solution Beacon, LLC. All Rights Reserved. 12 Install Oracle9i (9.0.1)... Modify init.ora and iflecbo.ora... Modify init.ora and iflecbo.ora...  Set the parameter _SYSTEM_TRIG_ENABLED to FALSE  change ifile entry in the init.ora to point to new location  db_block_buffers= (at least 2000)  shared_pool_size= (at least 150MB)  java_pool_size= (at least 20MB)  cluster_database=false  event="10619 trace name context forever, level 1"  event=”10932 trace name context level 32768“  event=”10933 trace name context level 512“  event=”10943 trace name context level 16384“

13. © 2005 Solution Beacon, LLC. All Rights Reserved. 13 Upgrade Oracle9i 9.0.1 db to 9.0.1.3 (1/03 upgraded to 9.0.1.4) 9.0.1 db to 9.0.1.3 (1/03 upgraded to 9.0.1.4)  note these are the Oracle9i binaries just installed -- not the Release11i db installed (it’s Oracle8i)  done via patch 2271678  DO NOT PERFORM any sql scripts, database update steps or POST INSTALL STEPS NOW -- DO THOSE AFTER THE 8.1.7.3 DATABASE IS UPGRADED

14. © 2005 Solution Beacon, LLC. All Rights Reserved. 14 Upgrade Apps 8i to 9i 8.1.7.3 db to 9.0.1.3 8.1.7.3 db to 9.0.1.3  include 9.0.1.3 home in ORACLE_HOME, PATH, LD_LIBRARY_PATH  start db restrict using modified init.ora  run upgrade script u0801070.sql (runs several hours)  run $ORACLE_HOME/rdbms/admin/utlip.sql because db was moving from 32 to 64 bit  shutdown Oracle9i db

15. © 2005 Solution Beacon, LLC. All Rights Reserved. 15 Upgrade Apps 8i to 9i 8.1.7.3 db to 9.0.1.3... 8.1.7.3 db to 9.0.1.3...  Re-adjusted init.ora file  Changed _system_trig_enabled = TURE  O7_DICTIONARY_ACCESSIBILITY=TRUE  Optimizer_features_enable = 9.0.1  restarted db  ran relevant scripts per Oracle9i migration manual  Jserver, XDK for java, Session Namespace, CORBA and OSE, jsp, oracle spatial, intermedia, oracle text  ran post install steps per patch 2271678

16. © 2005 Solution Beacon, LLC. All Rights Reserved. 16 Upgrade Apps 8i to 9i 8.1.7.3 db to 9.0.1.3... 8.1.7.3 db to 9.0.1.3...  Copy/Modify the listener.ora and tnsnames.ora to reflect Oracle9i locations  reset the following in init.ora set aq_tm_processes parameter = 1 set aq_tm_processes parameter = 1 set job_queue_processes = 2 set job_queue_processes = 2

17. © 2005 Solution Beacon, LLC. All Rights Reserved. 17 More 11i Adjustments for 9i Remove from applora.txt Remove from applora.txt_complex_view_merging_optimizer_mode_force_optimizer_undo_changes_or_expand_nvl_predicate_ordered_nested_loop_push_join_predicate_push_join_union_view_sort_elimination_cost_ratio_use_column_stats_for_functionalways_anti_joinalways_semi_joinoptimizer_percent_parallel

18. © 2005 Solution Beacon, LLC. All Rights Reserved. 18 More 11i Adjustments for 9i... Perform object maintenance Perform object maintenance – Compile all apps objects – Recreate grants and synonyms (adadmin) Increase MAXINSTANCES parameter Increase MAXINSTANCES parameter – to change this parameter, the database control file needs to be recreated, MAXINSTANCES parameter modified and the create control file sql executed shutdown/backup DB shutdown/backup DB

19. © 2005 Solution Beacon, LLC. All Rights Reserved. 19 Add RAC Raw files are required Raw files are required – shutdown db – cooked files must become raw – shared disk volumes must be configured for  database files  redologs  control files  existing cooked files copied to raw locations  the above was done with a combination veritas and dd commands

20. © 2005 Solution Beacon, LLC. All Rights Reserved. 20 Add RAC... Modify init.ora…node 1 Modify init.ora…node 1 – modified init.ora for RAC maxinstances=2 (customer had 2 nodes) maxinstances=2 (customer had 2 nodes) cluster_database=true cluster_database=true instance_name=mcprod1 instance_name=mcprod1 instance_number=1 instance_number=1 thread=1 thread=1 local_listener = "(address=(port=1522)(protocol=tcp)(ho st=sundb1))" local_listener = "(address=(port=1522)(protocol=tcp)(ho st=sundb1))"

21. © 2005 Solution Beacon, LLC. All Rights Reserved. 21 Add RAC... Change db file locations…node 1 Change db file locations…node 1 – create a sql script that will alter the location of the database files to reflect the new raw locations – startup the database mount – run the alter sql created above

22. © 2005 Solution Beacon, LLC. All Rights Reserved. 22 Add RAC... Modify init.ora…node 2 Modify init.ora…node 2 – ensure instance is shutdown – modified init.ora for RAC maxinstances=2 (customer had 2 nodes) maxinstances=2 (customer had 2 nodes) cluster_database=true cluster_database=true instance_name=mcprod2 instance_name=mcprod2 instance_number=2 instance_number=2 thread=2 thread=2 local_listener = "(address=(port=1522)(protocol=tcp)(h ost=sundb2))" local_listener = "(address=(port=1522)(protocol=tcp)(h ost=sundb2))"

23. © 2005 Solution Beacon, LLC. All Rights Reserved. 23 Add RAC... Configure listener.ora on node 1 Configure listener.ora on node 1 mcprod1 = (DESCRIPTION = (DESCRIPTION = (LOAD_BALANCE = OFF) (LOAD_BALANCE = OFF) (ADDRESS = (PROTOCOL = TCP)(HOST = sundb1)(PORT = 1522)) (ADDRESS = (PROTOCOL = TCP)(HOST = sundb1)(PORT = 1522)) ) SID_LIST_mcprod1 = (SID_LIST = (SID_LIST = (SID_DESC = (SID_DESC = (ORACLE_HOME = /u20/oracle/mc11idb) (ORACLE_HOME = /u20/oracle/mc11idb) (SID_NAME = mcprod1))) (SID_NAME = mcprod1)))

24. © 2005 Solution Beacon, LLC. All Rights Reserved. 24 Add RAC... Configure listener.ora on node 2 Configure listener.ora on node 2 mcprod2 = (DESCRIPTION = (DESCRIPTION = (LOAD_BALANCE = OFF) (LOAD_BALANCE = OFF) (ADDRESS = (PROTOCOL = TCP)(HOST = sundb2)(PORT = 1522)) (ADDRESS = (PROTOCOL = TCP)(HOST = sundb2)(PORT = 1522)) ) SID_LIST_mcprod2 = (SID_LIST = (SID_LIST = (SID_DESC = (SID_DESC = (ORACLE_HOME = /u20/oracle/mc11idb) (ORACLE_HOME = /u20/oracle/mc11idb) (SID_NAME = mcprod2))) (SID_NAME = mcprod2)))

25. © 2005 Solution Beacon, LLC. All Rights Reserved. 25 Add RAC... tnsnames.ora tnsnames.oramcprod2=(description= (address=(protocol=tcp)(host=sundb2)(port= 1522)) (address=(protocol=tcp)(host=sundb2)(port= 1522)) (connect_data=(service_name=mcprod))) (connect_data=(service_name=mcprod)))mcprod1= (description= (description= (address=(protocol=tcp)(host=sundb1)(port= 1522)) (address=(protocol=tcp)(host=sundb1)(port= 1522)) (connect_data=(service_name=mcprod))) (connect_data=(service_name=mcprod)))

26. © 2005 Solution Beacon, LLC. All Rights Reserved. 26 Add RAC... Tnsnames.ora… Tnsnames.ora…mcprod= (description=(load_balance=off)(failover=on) (description=(load_balance=off)(failover=on) (address_list= (address=(protocol=tcp)(host=sundb1)(port= 1522)) (address_list= (address=(protocol=tcp)(host=sundb1)(port= 1522)) (address=(protocol=tcp)(host=sundb2)(port= 1522)) ) (address=(protocol=tcp)(host=sundb2)(port= 1522)) ) (connect_data=(service_name=mcprod) (failover_mode=(type=select)(method=basic) (connect_data=(service_name=mcprod) (failover_mode=(type=select)(method=basic) (backup=mcprod2)))) (backup=mcprod2))))

27. © 2005 Solution Beacon, LLC. All Rights Reserved. 27 Add RAC... Start Instances and APPS Start Instances and APPS – started instance 1: startup parallel – added redo thread for instance 2 – started instance 2: startup parallel – started relevant apps processes on db tier – started relevant apps processes on app tier

28. © 2005 Solution Beacon, LLC. All Rights Reserved. 28 Add RAC... A virtual middle tier A virtual middle tier – Round robin dns was implemented because this customer had 2 apps boxes; Round robin dns requires adjustments to dns entries resulting in the ability to access either apps server with the same name; Later, this was replaced with Big IP Parallel Concurrent Processing Parallel Concurrent Processing – Concurrent processing was parallelized according the instructions in the system administration guide for Release 11i and MetaLink note 185489.1; In addition, a transaction manager should be setup via MetaLink note 240818.1

29. © 2005 Solution Beacon, LLC. All Rights Reserved. 29 Conclusion The process is complex The process is complex – Release 11i increases the complexity of implementing RAC – RAC increases the complexity of managing Release 11i (use oem and veritas volume mgr) – Attention getters Parallel concurrent processing Parallel concurrent processing Middle tier load balancing Middle tier load balancing Management of raw files Management of raw files

30. © 2005 Solution Beacon, LLC. All Rights Reserved. 30 References Sources used for this presentation Sources used for this presentation  metalink note 162091 – Interoperablility Notes: Oracle Applications Release 11i with Oracle9i Release 9.0.1  metalink note 183649.1 – Database upgrade from 32 to 64 bit (This note was written for Oracle8i - ensure you ignore items that no longer apply to Oracle9i  Oracle White paper by Ahmed Alomari of Oracle Corporation: Migrating Your E-Business Suite Single Instance to Real Application Clusters (RAC)  Oracle9i Database Migration Manual, Part Number A90191-02 (Manual upgrade section)  several tars

31. © 2005 Solution Beacon, LLC. All Rights Reserved. 31 Kevin Dahl kdahl@solutionbeacon.com www.solutionbeacon.com Hope you enjoyed this Release 11i session! Remember: KISS Keep It Simple and Supportable sm Q&A