1. Oracle 10g Advanced Performance Tuning Kyle Hailey KyleLF@gmail.com Delphix http://oraclemonitor.com - wait events docs http://ashmasters.com – tools S-ASH and ASHMON http://www.perfvision.com/ftp/class/2_hour_lvc - power points

2. Who is Kyle Hailey  1990 Oracle  90 support  92 Ported v6  93 France  95 Benchmarking  98 ST Real World Performance  2000 Dot.Com  2001 Quest  2002 Oracle OEM 10g  2006 Independent  2008 Embarcadero  DB Optimizer  2010 Delphix Success! First successful OEM design

3. 8 Copyright 2006 Kyle Hailey My Goal Simplify the information and empower the DBA

4. Launch: Pressure Midnight before January 28, 1986 Lives are on the line Thanks to Edward Tufte

5. Copyright 2006 Kyle Hailey 13 Pages Faxed

6. Copyright 2006 Kyle Hailey Original Engineering data only showed damage “damages at the hottest and coldest temperature” - management

7. Copyright 2006 Kyle Hailey Congressional Hearings Evidence

8. Copyright 2006 Kyle Hailey Clearer 1.Include successes 2.Mark Differences 3.Normalize same temp 4.Scale known vs unknown 55 65 75 60 70 80 4 812 4 812 30 40 50 35 45 X

9. Copyright 2006 Kyle Hailey Difficult  NASA Engineers Fail  Congressional Investigators Fail  Data Visualization is Difficult But … Lack of Clarity can be devastating

10. Solutions  Clear Identification  Know how to identify problems and issues  Access to details  Provide solutions and/or information to address the issues  Graphics  Easy understanding, effective communication and discussion

11. First Step: Graphics “The humans … are exceptionally good at parsing visual information, especially when that information is coded by color and/or _____.” Knowledge representation in cognitive science. Westbury, C. & Wilensky, U. (1998) motion

12. Why Use Graphics You can't imagine how many times I was told that nobody wanted or would use graphics … -- Jef Raskin, the creator of the Macintosh Infocus – (overhead projectors) sited a study that humans can parse graphical information 400,000 times faster than textual data

13. 18 Counties in US  3101 Counties in US  50 pages

14. “If I can't picture it, I can't understand it” Anscombe's Quartet IIIIIIIV xyxyxyxy 108.04109.14107.4686.58 86.9588.1486.7785.76 137.58138.741312.7487.71 98.8198.7797.1188.84 118.33119.26117.8188.47 149.96148.1148.8487.04 67.2466.1366.0885.25 44.2643.145.391912.5 1210.84129.13128.1585.56 74.8277.2676.4287.91 55.6854.7455.7386.89 Average 97.59 9 9 Standard Deviation 3.312.033.312.033.312.033.312.03 Linear Regression 1.33 - Albert Einstein

15. Graphics for Anscombe’s Quartet

16. What is a day in the life look like for a DBA who has performance issues? Tuning the DatabaseComplex Averages Anscombe's Quartet IIIIIIIV xyxyxyxy Average 97.59 9 9 Standard Deviation 3.312.033.312.033.312.033.312.03 Linear Regression 1.33

17. LOAD Top Activity SQL Events Events Sessions Get Details Click here Max CPU (yard stick) How Can We Open the Black Box?

18. OEMASHMON/SASH DB Optimizer Powerful - Identifies issues quickly and powerfullyPowerful - Identifies issues quickly and powerfully Interactive - Allows exploring the dataInteractive - Allows exploring the data Easy - Understandable by everyone, DBA, Dev and Managers !Easy - Understandable by everyone, DBA, Dev and Managers !

19. Copyright 2006 Kyle Hailey Ideas for TodayASH Sampling Waits AAS

20. Copyright 2006 Kyle Hailey Sections Day 1  New Ideas  Statspack  ASH  AAS  OEM 10g Day 2   Waits   Buffer Cache   IO   Redo   Enqueues   Shared Pool   SQL*Net Day 3   SQL Tuning http://oraclemonitor.com – wait documentation

21. Copyright 2006 Kyle Hailey Do You Want? Engineering Data?

22. Copyright 2006 Kyle Hailey Do You Want? Pretty Pictures

23. Copyright 2006 Kyle Hailey Do You Want? Clean and Clear ? ? ? ? ? ?

24. Copyright 2006 Kyle Hailey Imagine Trying to Drive your Car And is updated once and hour Or would you like it to look … Would you want your dashboard to look like :

25. Copyright 2006 Kyle Hailey Or This

26. Summary 1.Database - AAS  Profile database  Use wait interface and graphics  Identify machine, application, database or SQL 2.SQL - VST  Indexes, stats, execution path  Visual SQL Tuning

27. Bibliography http://www.simple-talk.com/sql/performance/designing-efficient-sql-a-visual-approach/ Refactoring SQL Applications – Stephane Faroult Troubleshooting Oracle Performance – Christian Antognini SQL Tuning – Dan Tow Cost-Based Oracle Fundamentals – Jonathan Lewis

28. END Copyright 2006 Kyle Hailey

29. When to Tune 1.Machine a) CPU  Response times skewed  100% CPU might be fine  Users wait in queue (run queue) => machine underpowered b) Memory  Paging  Wait times skewed (ex : latch free)  Erratic response times ( ex : ls ) 2.Oracle 1) Waits > CPU ?  tune waits 2) CPU > 100% ?  tune top CPU SQL 3) Else  It’s the application Oracle Load (AAS) Top Session Top Wait Top SQL SQL Detail Session Detail File Detail Object Detail Wait Detail Host AAS > #CPU AAS > 1 Waits > CPU CPU > Waits CPU Memory Memory

30. Machine Make sure the machine is healthy before tuning Oracle  CPU => use run queue, < 2 * #CPU  Memory => page out VMSTAT

31. Summary 1.Machine - vmstat  Memory, CPU (we can see IO response in Oracle) 2.Database - AAS  Use wait interface and graphics  Identify machine, application, database or SQL 3.SQL - VST  Indexes, stats, execution path  Visual SQL Tuning

32. How Can We Open the Black Box?OEMASHMON/SASH DB Optimizer Powerful - Identifies issues quickly and powerfullyPowerful - Identifies issues quickly and powerfully Interactive - Allows exploring the dataInteractive - Allows exploring the data Easy - Understandable by everyone, DBA, Dev and Managers !Easy - Understandable by everyone, DBA, Dev and Managers !