1. Average Session Load (ASL) The Golden Metric ? Kyle Hailey http://perfvision.com

2. #.2 Copyright 2006 Kyle Hailey In this Session The Holy Grail of Performance : ASL 1.ASL : stethoscope for Database Health  Tap into the heart beat of the database  Heart stopped - Hung?  Beating very slow - Idle ?  Beating fast - Heavily Loaded ? 2.Subcomponent Trinity  CPU  Waits  Time series 3.Extra: use Maximum CPU as Yardstick

3. #.3 Copyright 2006 Kyle Hailey Word of Wisdom My Goal is to cut out a lot of the noise and extraneous data and hone in on the essential Half of the game is knowing when to act and how much effort to put in

4. #.4 Copyright 2006 Kyle Hailey Idle Database  Value of proving the database is Idle  It’s the Databases Fault  How many times do you hear that?  Database Idle  No load on database  Database “performance” is fine  Under utilized  Problem lies elsewhere  Saved me time and stress many times

5. #.5 Copyright 2006 Kyle Hailey What’s the Database Doing  Often I want a quick and easy way to see what the database is doing  Is it working?  Is it blocked  How much is going on?  Is the database “healthy”?

6. #.6 Copyright 2006 Kyle Hailey Checking the Database  How do *you* check the Database Health?  Routine Exams?  Statspack?  1300 lines of data  Which lines  How many stats  Automated Alerts?  What do you set alerts on?  What if no alerts fire ? Are you ok?  Do alerts really tell you what’s happening?

7. #.7 Copyright 2006 Kyle Hailey What’s the Database Doing Whip out the stethoscope ASL

8. #.8 Copyright 2006 Kyle Hailey The Cult of ASL Once you’ve been initiated … there is no going back

9. #.9 Copyright 2006 Kyle Hailey Welcome to … The Cult of ASL  Magic Metric for Database Health Average Session Load (ASL) For those of you who are already members, welcome back

10. #.10 Copyright 2006 Kyle Hailey ASL  The Golden Metric  Powerful  Multidimensional  Indispensable

11. #.11 Copyright 2006 Kyle Hailey Average Session Load (ASL) goes under the guise of  Session Load  I often refer to it this way  Average Active Sessions (aas?)  The way I named it in OEM 10g Graphs  Centi-seconds per second (or secs/sec)  In the dark ages, before OEM 10g, waits were often measured as Centi-secs per sec

12. #.12 Copyright 2006 Kyle Hailey ASL – Average Session Load  Average Value  Averaged over 15 secs in OEM 10g  Time period could 5 minutes, an hour  Active Session Only  Active sessions put load the database  Inactive Sessions don’t put load  Except for memory usage  Active sessions are  Sessions in a Call  Started a SQL Statement but hasn’t returned yet  DBWR writing blocks out

13. #.13 Copyright 2006 Kyle Hailey ASL Calculations ASL = DB TIME / Time Period DB Time (DBT) = sum over all sessions of time in the states   CPU   Wait DB TIME (10g) = DB TIME = select value from v$sysstat where name = ‘DB time’; select value from v$sysstat where name = ‘DB time’; Select sum(time_waited) from v$system_event where event not in (... idle events …); + Select value from v$sysstat where name = ‘CPU used by this session’; Select sum(time_waited) from v$system_event where event not in (... idle events …); + Select value from v$sysstat where name = ‘CPU used by this session’; ‘DB time’

14. #.14 Copyright 2006 Kyle Hailey ASL – sources ASL can be found from or derived from  10g  OEM  V$systat ‘DB time’ / elapsed time  9i  v$system_event  $sysstat  This works Oracle 7 to 10g (probably 11)

15. #.15 Copyright 2006 Kyle Hailey ASL – 9i  Data  V$system_event  Sum wait times  Non-idle waits  V$sysstat  CPU used by this session  ASL= (sums(wait)-cpu)/elapsed time  ASL(CPU)=cpu/elasped  ASL(wait)= sum(waits)/elapsed  Produces  Session time / elapsed time  Session cent-secs/sec  Session secs/ sec  Avg Session Load (ASL)

16. #.16 Copyright 2006 Kyle Hailey ASL in OEM DB Home Page

17. #.17 Copyright 2006 Kyle Hailey Calculating ASL : Statspack  Uses  v$session_wait  v$sysstat  Look for  Top 5 Timed Events  Elapsed Time  cpu_count helpful Seconds in Wait / Elapsed time = ASL

18. #.18 Copyright 2006 Kyle Hailey Use Statspack to Find Waits Statspack Cheat Sheet:  Install  Connect as SYSDBA  @?/rdbms/admin/spcreate.sql  Run  Exec statspack.snap;  Generate Reports  @?/rdbms/admin/spreport.sql

19. #.19 Copyright 2006 Kyle Hailey Statspack Trusty statspack report  Elapased Time  Check “Top 5 Timed Events” Start at line 52 of about 1300!

20. #.20 Copyright 2006 Kyle Hailey Elapsed Time STATSPACK report for DB Name DB Id Instance Inst Num Release RAC Host ------- ----------- -------- -------- ---------- --- ------- LABSF03 1420044432 labsf03 1 10.1.0.2.0 NO labsfr Snap Id Snap Time Sessions Curs/Sess --------- ------------------ -------- --------- Begin Snap: 1 03-Apr-06 12:34:06 18 5.6 End Snap: 2 03-Apr-06 12:34:36 18 4.8 Elapsed: 1.00 (mins) STATSPACK report for DB Name DB Id Instance Inst Num Release RAC Host ------- ----------- -------- -------- ---------- --- ------- LABSF03 1420044432 labsf03 1 10.1.0.2.0 NO labsfr Snap Id Snap Time Sessions Curs/Sess --------- ------------------ -------- --------- Begin Snap: 1 03-Apr-06 12:34:06 18 5.6 End Snap: 2 03-Apr-06 12:34:36 18 4.8 Elapsed: 1.00 (mins)

21. #.21 Copyright 2006 Kyle Hailey Used CPU Time and Wait Time  Look at Top 5 Timed Events Top 5 Timed Events ~~~~~~~~~~~~~~~~~~ % Total Event Waits Time (s) Call Time --------------------- --------- --------- ------- buffer busy waits 2,748 250 78.72 CPU time 32 10.16 free buffer waits 1,588 15 4.63 write complete waits 10 8 2.51 log buffer space 306 5 1.51 Top 5 Timed Events ~~~~~~~~~~~~~~~~~~ % Total Event Waits Time (s) Call Time --------------------- --------- --------- ------- buffer busy waits 2,748 250 78.72 CPU time 32 10.16 free buffer waits 1,588 15 4.63 write complete waits 10 8 2.51 log buffer space 306 5 1.51

22. #.22 Copyright 2006 Kyle Hailey Example  CPU + WAITS CPU = 32 WAITS = 250 + 15 + 8 + 5 = 278 secs  Elapsed Time = 60 secs  (32 + 278) user secs / 60 secs = 5.1 average session load ( 4.6 waiting 0.5 on CPU ) Top 5 Timed Events Event Time (s) ----------------- ----- buffer busy waits 250 CPU time 32 free buffer waits 15 write complete waits 8 log buffer space 5 Top 5 Timed Events Event Time (s) ----------------- ----- buffer busy waits 250 CPU time 32 free buffer waits 15 write complete waits 8 log buffer space 5

23. #.23 Copyright 2006 Kyle Hailey Available CPU – init.ora Statspack 10g shows # of CPUS Statspack 9i # of CPUs SQLPLUS> show parameters cpu_count NAME TYPE VALUE ------------------ -------- ---------- cpu_count integer 2 SQLPLUS> show parameters cpu_count NAME TYPE VALUE ------------------ -------- ---------- cpu_count integer 2

24. #.24 Copyright 2006 Kyle Hailey Available CPU vs ASL Far above available CPU => problem Plenty of Free CPU => wait bottleneck 4.6 waiting 0.5 on CPU 4.6 waiting 0.5 on CPU ASL = 5.1 # of CPU = 2 ASL = 5.1 # of CPU = 2

25. #.25 Copyright 2006 Kyle Hailey ASL – Primary Purpose  Answers the Question Is the database idle ? Active? How Active?

26. #.26 Copyright 2006 Kyle Hailey ASL < 1 Database is not blocked ASL ~= 0 Database basically idle ASL < # of CPUs Extra CPU to be had Database is probably not blocked ASL > # of CPUs Could have performance problems  ASL >> # of CPUS There is a bottleneck

27. #.27 Copyright 2006 Kyle Hailey Going Farther with ASL  ASL can tell you a lot  But it’s components tell you much more  To go farther need the components of ASL  CPU  How many CPUs (max CPU available)  Wait  Which waits  Value over time

28. #.28 Copyright 2006 Kyle Hailey Components of ASL DB Home Page Performance Page ASL Point in Time ASL over Time

29. #.29 Copyright 2006 Kyle Hailey EM DB Home Page

30. #.30 Copyright 2006 Kyle Hailey Performance Page

31. #.31 Copyright 2006 Kyle Hailey Performance Page

32. #.32 Copyright 2006 Kyle Hailey ASL – Performance Page

33. #.33 Copyright 2006 Kyle Hailey OEM 10g - ASL ASL is the top of the curve 3 1.9 2

34. #.34 Copyright 2006 Kyle Hailey OEM 10g - CPU CPU

35. #.35 Copyright 2006 Kyle Hailey OEM 10g - Waits Waits

36. #.36 Copyright 2006 Kyle Hailey OEM 10g – CPU vs Wait Waits CPU

37. #.37 Copyright 2006 Kyle Hailey OEM 10g – Max CPU Maximum Possible CPU

38. #.38 Copyright 2006 Kyle Hailey OEM 10g Zoom-In Available CPU CPU + WAIT

39. #.39 Copyright 2006 Kyle Hailey OEM 10g – get to work ! Relax Get to Work! Looks OK But …

40. #.40 Copyright 2006 Kyle Hailey Calculating ASL

41. #.41 Copyright 2006 Kyle Hailey ASL Calculations ASL = DB TIME / Time Period DB TIME (10g) = select value from v$sysstat where name = ‘DB time’; select value from v$sysstat where name = ‘DB time’; But there is another way …

42. #.42 Copyright 2006 Kyle Hailey ASL alternative Calculation  active sessions  Count Active sessions over interval  average by interval  Less accurate, but surprisingly close  v$session_wait (or v$active_session_history)  Wait_time > 0 = ON CPU  Filter out idle events  9i or lower, join to v$session  status='ACTIVE'  type='USER‘  10g v$sessions has all the columns

43. #.43 Copyright 2006 Kyle Hailey Two Sources comparison  V$system_event & v$sysstat  Indirect measure via time spend in DB  Accurate Measure of time counters  Values can lag (especially CPU)  v$session_wait  Direct measure of # of sessions  Closer to “real time”  statistical approximation via samples

44. #.44 Copyright 2006 Kyle Hailey ASL in OEM DB Home Performance Page Top Activity Page Same Chart but calculated differently 1 2

45. #.45 Copyright 2006 Kyle Hailey ASL – Performance Page v$sysstat v$system_event ASL = ΔDBT / ΔTime ASL(tn)= ( DBT(tn) - DBT(t0) ) / (tn-t0) ASL 1

46. #.46 Copyright 2006 Kyle Hailey ASL – Top Activity Page active sessions(ti) n Σ 0 ASL(tn) = / n 7654321 t0 t1 2 n = # of samples

47. #.47 Copyright 2006 Kyle Hailey Active Average Sessions = Top Activty ? active sessions(ti) n Σ 0 ASL(tn) = / n n = # of samples ASL = ΔDBT / ΔTime ASL(tn)= ( DBT(tn) - DBT(t0) ) / (tn-t0) Performance Page : Average Active Sessions Top Activity = ?

48. #.48 Copyright 2006 Kyle Hailey DB TIME = area under the curve ΔDBTΔTime ASL = ΔDBT/ ΔTime ΔDBT = DBT(tn) – DBT(t0) = Δt ΔDBT = DBT(tn) – DBT(t0) = active sessions(ti) * Δt n Σ 0 ΔDBT ΔDBT = Area under curve 7654321 t0 tn t0 tn

49. #.49 Copyright 2006 Kyle Hailey ASL – Top Activity Page ΔTime = n Σ 0 Δt / Δt active sessions(ti) * Δt n Σ 0 ASL(t1) = n Σ 0 Δt active sessions(ti) n Σ 0 ASL(t1) = / n Σ 0 1 active sessions(ti) n Σ 0 ASL(t1) = / n n = # of samples Δt active sessions(ti) * Δt n Σ n ASL(t1) = / ΔTime ΔDBTΔTime ASL = ΔDBT/ ΔTime ΔDBT = Δt ΔDBT = active sessions(ti) * Δt n Σ 0

50. #.50 Copyright 2006 Kyle Hailey ASL – Top Activity Page 7654321 t0 tn / active sessions(ti) n Σ 0 ASL(t1) = n

51. #.51 Copyright 2006 Kyle Hailey Samples VS Counters Counters Samples Slight Lags v$system_event v$active_session_history

52. #.52 Copyright 2006 Kyle Hailey The Power ASH gives ASL DB Home Performance Page Top Activity Page

53. #.53 Copyright 2006 Kyle Hailey ASH in OEM Top Activity gives more information

54. #.54 Copyright 2006 Kyle Hailey Top Activity : Based on ASH missing ThanksToASH

55. #.55 Copyright 2006 Kyle Hailey Top Activity : ASH Dimensions

56. #.56 Copyright 2006 Kyle Hailey ASL – %Session Time Issue Shown in % DB Time Missing % Session Time

57. #.57 Copyright 2006 Kyle Hailey Top Activity: ASH Sessions Many Users Active On Performance Page, no way to tell how many users But Top Activity Page fixes that

58. #.58 Copyright 2006 Kyle Hailey Top Activity: ASH Sessions Two Users Active

59. #.59 Copyright 2006 Kyle Hailey SQL and Session DB Home Performance Top Activity SQL Session

60. #.60 Copyright 2006 Kyle Hailey Session : ASH Activity

61. #.61 Copyright 2006 Kyle Hailey SQL : ASH Activity

62. #.62 Copyright 2006 Kyle Hailey Getting the Most  Need to know the System’s Profile  What your application is like  Data Warehouse  OLTP  Typical load  Once you get to know it you can see anomalies  Is ASL near 0 when it should be higher  Is that Data Warehouse query running normal  Do you know what it looks like?  Is there an unusual bottleneck

63. #.63 Copyright 2006 Kyle Hailey Knowing your DB Profile

64. #.64 Copyright 2006 Kyle Hailey When to tune General rules of Thumb  Waits >> CPU  CPU > Max CPU

65. #.65 Copyright 2006 Kyle Hailey Waits > CPU

66. #.66 Copyright 2006 Kyle Hailey CPU > Max CPU

67. #.67 Copyright 2006 Kyle Hailey CPU > Max CPU

68. #.68 Copyright 2006 Kyle Hailey Getting More out of ASL DB Home Performance Top Activity SQL Session Session

69. #.69 Copyright 2006 Kyle Hailey In resume  ASL is simple and Powerful  ASL’s components are even more powerful  CPU  WAIT  Value over Time  Use # of CPUs as a yardstick  Know your application load profile to see anomalies  Data warehouse  OLTP  Heavy Load  Light Load