ASH – Active Session History Feel the Power Kyle Hailey http://perfvision.com

Why should you care? Because ASH can Change your life … 10g immediately Accessible Geeks: Via scripts in SQL Managers : Graphical OEM Stuck on Oracle 7,8,9 ? … my apologies because 10g rocks, but … no worries, the data is accessible for you too via scripts http://perfvision.com/sash.php Copyright 2006 Kyle Hailey

Revolution and an Evolution Oracle 6 … ie the dark ages … there was once the Cache Buffer Hit Ratio Oracle 7 … turned the lights on … Wait Events … hallelujah I can see the light Oracle 10g … ASH has landed (like the wheels on the suitcases) Copyright 2006 Kyle Hailey

Sampling New Paradigm Clear Vision of the future Cheaper Use new sampling technology Lets go of the need to know 100% Statistical approximation Cheaper Richer Multidimensional Data Identifies SQL Sessions Objects Machine User Sampling is a leap in performance tuning technology. It took a clear vision of the future and a bit of a leap of faith to let go of the compulsive need to have exact measurements and instead embrace sampling a technology that statistically approximates the same value with little loss of accuracy but at the same time brings a much great wealth of information necessary for analyzing the performance of a database. Copyright 2006 Kyle Hailey

Identify Players before ASH Sessions v$sesstat, v$session_event # sessions x (# wait events + statistics) Example (150 x (800+200) = 150,000 ) SQL v$sql Could be 10000s Takes out latches that compete with other sql executions Objects V$segstat 9i+ Could be 1000s of objects Also files v$filestat Expensive ! Copyright 2006 Kyle Hailey

ASH Intelligence for the new Millennium Intelligently Collects Data Samples once a second Collects active sessions only Collects Session/SQL/Wait More activity, more data collected Less activity, less data collected Those old methods collected everything Obfuscated the problem, too many statistics too late Costly Too Granular – once an hour ?! Give me a break Collects fine grain multidimensional data Copyright 2006 Kyle Hailey

ASH Fields SAMPLE_TIME TIMESTAMP(3) SESSION_ID NUMBER SQL> v$active_session_history Name Null? Type SAMPLE_ID NUMBER ----------------------------------------- -------- ---------------------------- SAMPLE_TIME TIMESTAMP(3) SESSION_ID NUMBER SESSION_SERIAL# NUMBER USER_ID NUMBER SQL_ID VARCHAR2(13) SQL_CHILD_NUMBER NUMBER SQL_PLAN_HASH_VALUE NUMBER SQL_OPCODE NUMBER SERVICE_HASH NUMBER SESSION_TYPE VARCHAR2(10) SESSION_STATE VARCHAR2(7) QC_SESSION_ID NUMBER QC_INSTANCE_ID NUMBER EVENT VARCHAR2(64) EVENT_ID NUMBER EVENT# NUMBER SEQ# NUMBER P1 NUMBER P2 NUMBER P3 NUMBER WAIT_TIME NUMBER TIME_WAITED NUMBER CURRENT_OBJ# NUMBER CURRENT_FILE# NUMBER CURRENT_BLOCK# NUMBER PROGRAM VARCHAR2(48) MODULE VARCHAR2(48) ACTION VARCHAR2(32) CLIENT_ID VARCHAR2(64 Copyright 2006 Kyle Hailey

v$active_session_history SAMPLE_ID NUMBER SAMPLE_TIME TIMESTAMP(3) When SESSION_ID NUMBER SESSION_SERIAL# NUMBER USER_ID NUMBER SERVICE_HASH NUMBER SESSION_TYPE VARCHAR2(10) PROGRAM VARCHAR2(64) MODULE VARCHAR2(48) ACTION VARCHAR2(32) CLIENT_ID VARCHAR2(64) Session State SESSION_STATE VARCHAR2(7) WAIT_TIME NUMBER EVENT VARCHAR2(64) EVENT_ID NUMBER EVENT# NUMBER SEQ# NUMBER P1 NUMBER P2 NUMBER P3 NUMBER WAIT_TIME NUMBER TIME_WAITED NUMBER CURRENT_OBJ# NUMBER CURRENT_FILE# NUMBER CURRENT_BLOCK# NUMBER0 Wait WAIT_TIME is time of last wait, > 0 , means on CPU TIME_WAITED is the actual wait time SQL SQL_ID VARCHAR2(13) SQL_CHILD_NUMBER NUMBER SQL_PLAN_HASH_VALUE NUMBER SQL_OPCODE NUMBER QC_SESSION_ID NUMBER QC_INSTANCE_ID NUMBER Duration TIME_WAITED NUMBER Copyright 2006 Kyle Hailey

Primary Fields of ASH SAMPLE_TIME Time SESSION_ID Session Activity : Who, What, When, How When SAMPLE_TIME Time SESSION_ID Who Session SESSION_STATE State How EVENT Wait What SQL_ID SQL Copyright 2006 Kyle Hailey

Amazing things YOU can do Top SQL SQL ID Plan Child# Top Consumers Session User Object Module.Action Program Service Client Wait Top Resources CPU Waits Event (800*) I/O File Block Time X X And Aggregated over any time Period (32 columns in ASH 10.1) Copyright 2006 Kyle Hailey

Multi-dimensional SQL Statistics Waits Sessions 06/05/08 Program User F1qcyh20550cf fj6gjgsshtxyx 0cjsxw5ndqdbc 8t8as9usk11qw dr1rkrznhh95b 10dkqv3kr8xa5 38zhkf4jdyff4 298wmz1kxjs1m Package Procedure Plan Child # Statistics Waits CPU Enq: TX – row lock contention SQL*Net break/reset to client db file scattered read db file sequential read Sessions 25 34 36 38 45 63 65 87 Program Sqlplus Toad User Sys Scott System Application Service GL OE IO 06/05/08

Groupings – Top Consumer SESSION_ID SESSION_SERIAL# (identify SID reuse) SESSION_TYPE (FOREGROUND,BACKGROUND) USER_ID (SYS, SYSTEM, SCOTT etc) SERVICE_HASH (OE,GL,HR) MODULE.ACTION (PLSQL tagging) CLIENT_ID (identifying users in session pool) PROGRAM (SQL, JDBC, Forms etc) SQL_ID QC_SESSION_ID (Query Coordinator) QC_INSTANCE_ID (RAC) EVENT + P1, P2, P3 CURRENT_OBJ# CURRENT_FILE# CURRENT_BLOCK# Copyright 2006 Kyle Hailey

Top CPU Session Top CPU consuming Session in last 5 minutes Select session_id, count(*) from v$active_session_history where session_state= ‘ON CPU‘ and SAMPLE_TIME > sysdate – (5/(24*60)) group by session_id order by count(*) desc; Who is the rogue session ? Copyright 2006 Kyle Hailey

Results Top CPU Session SESSION_ID COUNT(*) ---------- ---------- 257 299 263 62 256 32 264 9 277 3 258 1 Select session_id, session_serial# , count(*) , round((count(*)*100)/(5*60),0) "%", lpad('*',round((count(*)*10)/(5*60),0),'*') "Bar" from v$active_session_history where session_state= 'ON CPU' and SAMPLE_TIME > sysdate - (5/(24*60)) group by session_id, session_serial# order by count(*) desc / Copyright 2006 Kyle Hailey

CPU with Bars Bar shows 10% increments SESSION_ID COUNT(*) % Bar ---------- ---------- ---------- ------------ 257 299 99 |**********| 263 62 21 |** | 256 32 11 |* | 264 9 3 | | 277 3 1 | | 258 1 0 | | 280 1 0 | | Graphics – humans can process 400,000 times as much data graphically as textually Think of a map of the US and all the counties with birth rates colored from low to high. It’s a massive about of information textually but it can all be seen on one page of a map Select session_id, session_serial# , count(*) , round((count(*)*100)/(5*60),0) "%", lpad('*',round((count(*)*10)/(5*60),0),'*') "Bar" from v$active_session_history where session_state= 'ON CPU' and SAMPLE_TIME > sysdate - (5/(24*60)) group by session_id, session_serial# order by count(*) desc / Bar shows 10% increments Copyright 2006 Kyle Hailey

Top Waiting Session Top Waiting Session in last 5 minutes Select session_id, count(*) from v$active_session_history where session_state=‘WAITING’ and SAMPLE_TIME > SYSDATE - (5/(24*60)) group by session_id order by count(*) desc; Copyright 2006 Kyle Hailey

Top Waiting Session Results SESSION_ID COUNT(*) ---------- ---------- 272 224 254 8 249 5 276 5 277 4 270 1 Copyright 2006 Kyle Hailey

Top SQL from ASH select ash.SQL_ID , sum(decode(ash.session_state,'ON CPU',1,0)) "CPU", sum(decode(ash.session_state,'WAITING',1,0)) - sum(decode(ash.session_state,'WAITING', decode(en.wait_class, 'User I/O',1,0),0)) "WAIT" , sum(decode(ash.session_state,'WAITING', decode(en.wait_class, 'User I/O',1,0),0)) "IO" , sum(decode(ash.session_state,'ON CPU',1,1)) "TOTAL" from v$active_session_history ash, v$event_name en where SQL_ID is not NULL and en.event#=ash.event# group by sql_id order by sum(decode(session_state,'ON CPU',1,1)) desc select ash.SQL_ID , sum(decode(ash.session_state,'ON CPU',1,0)) "CPU", sum(decode(ash.session_state,'WAITING',1,0)) - sum(decode(ash.session_state,'WAITING', decode(en.wait_class, 'User I/O',1,0),0)) "WAIT" , sum(decode(ash.session_state,'WAITING', decode(en.wait_class, 'User I/O',1,0),0)) "IO" , sum(decode(ash.session_state,'ON CPU',1,1)) "TOTAL" from v$active_session_history ash, v$event_name en where SQL_ID is not NULL and en.event#=ash.event# group by sql_id order by sum(decode(session_state,'ON CPU',1,1)) desc Copyright 2006 Kyle Hailey

Top SQL from ASH Results SQL_ID CPU WAITING IO TOTAL ------------- ---------- ---------- ---------- ---------- 4c1xvq9ufwcjc 23386 0 0 23386 6wjw6rz5uvbp3 99 0 23 122 968dm8hr9qd03 97 0 22 119 938jp5gasmrah 90 0 25 115 cv8xnv81kf582 42 0 9 51 6p9bzu19v965k 21 0 0 21 5zu8pxnun66bu 15 0 0 15 db2jr13nup72v 9 0 0 9 7ks5gnj38hghv 8 0 0 8 Copyright 2006 Kyle Hailey

Top Session select ash.session_id, ash.session_serial#, ash.user_id, ash.program, sum(decode(ash.session_state,'ON CPU',1,0)) "CPU", sum(decode(ash.session_state,'WAITING',1,0)) - sum(decode(ash.session_state,'WAITING', decode(en.wait_class,'User I/O',1, 0 ), 0)) "WAITING" , decode(en.wait_class,'User I/O',1, 0 ), 0)) "IO" , sum(decode(session_state,'ON CPU',1,1)) "TOTAL" from v$active_session_history ash, v$event_name en where en.event# = ash.event# group by session_id,user_id,session_serial#,program order by sum(decode(session_state,'ON CPU',1,1)) select ash.session_id, ash.session_serial#, ash.user_id, ash.program, sum(decode(ash.session_state,'ON CPU',1,0)) "CPU", sum(decode(ash.session_state,'WAITING',1,0)) - sum(decode(ash.session_state,'WAITING', decode(en.wait_class,'User I/O',1, 0 ), 0)) "WAITING" , decode(en.wait_class,'User I/O',1, 0 ), 0)) "IO" , sum(decode(session_state,'ON CPU',1,1)) "TOTAL" from v$active_session_history ash, v$event_name en where en.event# = ash.event# group by session_id,user_id,session_serial#,program order by sum(decode(session_state,'ON CPU',1,1)) Copyright 2006 Kyle Hailey

Top Session Results SESSION_ID SERIAL# USER_ID PROGRAM CPU WAITING IO ---------- ------- ---------- ------------------------- ------- ---------- ---------- 247 61970 1 sqlplus 11698 0 0 277 1 0 oracle@labsfrh903 (LGWR) 14 21 0 276 1 0 oracle@labsfrh903 (CKPT) 19 10 0 278 1 0 oracle@labsfrh903 (DBW0) 29 0 0 280 1 0 oracle@labsfrh903 (PMON) 19 0 0 254 22617 5 Executor.exe 13 0 3 255 12877 5 Executor.exe 11 0 5 257 33729 5 Executor.exe 15 0 1 255 13417 5 Executor.exe 14 0 2 Copyright 2006 Kyle Hailey

Top Session w/ Username select decode(nvl(to_char(s.sid),-1),-1,'DISCONNECTED','CONNECTED') "STATUS", topsession.session_id "SESSION_ID", u.name "NAME", topsession.program "PROGRAM", max(topsession.CPU) "CPU", max(topsession.WAITING) "WAITING", max(topsession.IO) "IO", max(topsession.TOTAL) "TOTAL" from ( {previous query} ) topsession, v$session s, user$ u where u.user# =topsession.user_id and /* outer join to v$session because the session might be disconnected */ topsession.session_id = s.sid (+) and topsession.session_serial# = s.serial# (+) group by topsession.session_id, topsession.session_serial#, topsession.user_id, topsession.program, s.username,s.sid,s.paddr,u.name order by max(topsession.TOTAL) desc Copyright 2006 Kyle Hailey

Top Session Finding a Rogue User STATUS SESSION_ID NAME PROGRAM CPU WAITING IO --------------- ---------- ---------- ------------------------- ----- ---------- ---- CONNECTED 247 CPU_Monger ChMgr304.exe 11704 0 0 CONNECTED 277 SYS oracle@labsfrh903 (LGWR) 14 19 0 CONNECTED 278 SYS oracle@labsfrh903 (DBW0) 29 0 0 CONNECTED 276 SYS oracle@labsfrh903 (CKPT) 18 9 0 CONNECTED 280 SYS oracle@labsfrh903 (PMON) 20 0 0 DISCONNECTED 255 SYSTEM Executor.exe 11 4 5 DISCONNECTED 257 SYSTEM Executor.exe 13 0 3 DISCONNECTED 255 SYSTEM Executor.exe 14 0 2 Copyright 2006 Kyle Hailey

Many Ways to Attack Problems Confusing How to Attack the problem? Top SQL? Top wait for that SQL? Top Waiting Session ? Top Waits for that Session Top Waits for Database? Top Session waiting for that wait Top SQL for that wait Solution - Graphics Copyright 2006 Kyle Hailey

Mining Data is Non Trivial Many Dimensions to consider Constantly Varying time frames Luckily 10g automates all of this functionality. collects all of the data analyses all the wait events reports on bottlenecks supplies solutions Graphics add tremendous ease and power Copyright 2006 Kyle Hailey

OEM Perf Page Copyright 2006 Kyle Hailey

ASH Sampling Cheap Cost Size With PL/SQL it’s less that 1 % CPU with 10 average active sessions DMA - direct memory attach to SGA Using C code 200x Cheaper ASH in 10g should be even cheaper than DMA because it’s done in the kernel Size Circular Buffer - 1M to 128M (~2% of SGA) Avg row around 150bytes 3600 secs in an hour ~ ½ Meg per Active Session per hour Copyright 2006 Kyle Hailey

How ASH works V$session_wait+ v$session + extras MMON/MMNL v$active_session_history Only writes out 1 in 10 rows via direct path inserts wrh$_active_session_history MMON - manageability monitor, AWR M000 – MMON background slave, runs snapshots MMNL - MMON lite, flushes ASH to disk Copyright 2006 Kyle Hailey

ASH buffer Select reads backwards No latching Index on time No read consistency Insert point Insert one direction Touch up wait times Copyright 2006 Kyle Hailey

Family of ASH Tables Real time v$session ~2 hours (before 10g v$session_wait) Last 10 waits v$session_wait_history ~2 hours v$active_session_history 1 week DBA_HIST_ACTIVE_SESS_HISTORY wrh$active_session_history Copyright 2006 Kyle Hailey

Oradebug Dump to trace file SQL> oradebug dump ashdump 5 SQL> Alter session set events ‘immediate tracename ashdump level 5’; level 5 = # of minutes loader file rdbms/demo/ashldr.ctl SQL> oradebug dump ashdump 5 SQL> Alter session set events ‘immediate trace name ashdump level 5’; Copyright 2006 Kyle Hailey

INIT.ORA statistics_level = Typical (default) PARAMETER SESSION_VALUE INSTANCE_VAL ----------------------- ---------- ------------ _ash_sampling_interval 1000 1000 milliseconds _ash_size 1048618 1048618 ASH buffer size _ash_enable TRUE TRUE Turn on/off ASH sampling, flushing and the V$ views on ASH _ash_disk_write_enable TRUE TRUE Flush to disk _ash_disk_filter_ratio 10 10 write 1 in 10 points _ash_sample_all FALSE FALSE Sample including idle waits ASH _ash_enable = false; [ A dynamic parameter will turn off ASH sampling, flushing and the V$ views on ASH ] ADDM _addm_auto_enable = false; [ A dynamic parameter to turn off automatic ADDM runs after every AWR snapshot ] AWR "_swrf_mmon_flush" = FALSE ; AWR metrics "_swrf_mmon_metrics" = FALSE ; METRICS DB Feature Usage "_swrf_mmon_dbfus" = FALSE ; DB Feature Usage "_swrf_on_disk_enabled" = FALSE ; disable all (on disk, including manual) AWR operations: Copyright 2006 Kyle Hailey

10.2 ASH Extras Blocking Session ! Parameter Names Wait Grouping XID BLOCKING_SESSION_STATUS BLOCKING_SESSION_SERIAL# RAC event Fixup Plan Hash Fix up 10gR1 - during parsing no plan, good way to find parsing problems 10gR2 – get plan hash fixup – good but lose some ability to find parsing problems Parameter Names P1TEXT , P2TEXT, P3TEXT Wait Grouping WAIT_CLASS WAIT_CLASS_ID XID FORCE_MATCHING_SIGNATURE Cursor sharing 6/2/2018 33 33

10.2.0.3 ASH Extras PLSQL_ENTRY_OBJECT_ID PLSQL_ENTRY_SUBPROGRAM_ID PLSQL_OBJECT_ID PLSQL_SUBPROGRAM_ID ALL_PROCEDURES where object_id = plsql_object_id and subprogram_id = plsql_subprogram_id 6/2/2018 34 34

PLSQL Tracking Package/Procedure/SQL tracking 10.2.0.3 calling_code SQL_ID COUNT(*) ----------------------------------------------- ---------- ORDERENTRY.NEWORDER 0uuqgjq7k12nf 258 ORDERENTRY.NEWCUSTOMER 0bzhqhhj9mpaa 262 ORDERENTRY.BROWSEANDUPDATEORDERS 41zu158rqf4kf 301 ORDERENTRY.NEWORDER 0yas01u2p9ch4 569 PARSE_SAME 3vjxpmhhzngu4 874 ORDERENTRY.BROWSEANDUPDATEORDERS 05s4vdwsf5802 1669 set linesize 120 col entry_package for a25 col entry_procedure for a25 col cur_package for a25 col cur_procedure for a25 col calling_code for a70 select procs1.object_name || decode(procs1.procedure_name,'','','.')|| procs1.procedure_name ||' '|| decode(procs2.object_name,procs1.object_name,'',procs2.object_name) || decode(procs2.procedure_name,procs1.procedure_name,'', decode(procs2.procedure_name,'','',null,'','.')||procs2.procedure_name) "calling_code", sql_id, count(*) from v$active_session_history ash, all_procedures procs1, all_procedures procs2 where ash.PLSQL_ENTRY_OBJECT_ID = procs1.object_id (+) and ash.PLSQL_ENTRY_SUBPROGRAM_ID = procs1.SUBPROGRAM_ID (+) and ash.PLSQL_OBJECT_ID = procs2.object_id (+) and ash.PLSQL_SUBPROGRAM_ID = procs2.SUBPROGRAM_ID (+) and ash.sample_time > sysdate - &minutes/(60*24) group by procs1.object_name, procs1.procedure_name, procs2.object_name, procs2.procedure_name,sql_id order by count(*) / select object_name package||.|| procedure_name, sql_id, count(*) from v$active_session_history ash, all_procedures procs where ash.PLSQL_ENTRY_OBJECT_ID = procs.object_id and ash.PLSQL_ENTRY_SUBPROGRAM_ID = procs.SUBPROGRAM_ID 6/2/2018 35 35

11.1g ASH extras SQL Elapsed Operation bit vector – non timed ops  SQL_EXEC_ID  SQL_EXEC_START  IN_CONNECTION_MGMT  IN_PARSE  IN_HARD_PARSE  IN_SQL_EXECUTION  IN_PLSQL_EXECUTION  IN_PLSQL_RPC  IN_PLSQL_COMPILATION  IN_JAVA_EXECUTION  IN_BIND  IN_CURSOR_CLOSE SQL Row Source SQL_PLAN_LINE_ID  SQL_PLAN_OPERATION  SQL_PLAN_OPTIONS Which instance requested block?  REMOTE_INSTANCE# ETC Recursive SQL CURRENT_ROW#   EVENT#   QC_SESSION_SERIAL#  CONSUMER_GROUP_ID  FLAGS TOP_LEVEL_SQL_ID TOP_LEVEL_SQL_OPCODE Copyright 2006 Kyle Hailey 36

11.2g ASH extras Statistics TM_DELTA_TIME TM_DELTA_CPU_TIME TM_DELTA_DB_TIME DELTA_TIME DELTA_READ_IO_REQUESTS DELTA_WRITE_IO_REQUESTS DELTA_READ_IO_BYTES DELTA_WRITE_IO_BYTES DELTA_INTERCONNECT_IO_BYTES PGA_ALLOCATED TEMP_SPACE_ALLOCATE Copyright 2006 Kyle Hailey 37

ASH Across Versions When Who SQL ASH instrumentation is exploding across the versions Waits Statistics Blocker 10.1 10.2 10.2.0.3 11.1 11.2 Copyright 2006 Kyle Hailey

How Many Active Sessions? How much data does ASH Collect ? 1 CPU means max 1 Avg Active Session unless there is a bottleneck Big site examples: Oracle 4 way RAC internal apps 10,000 connected, 200 active One Site 3000 connected, 30 Active Site 12,000 connected, 100 active Copyright 2006 Kyle Hailey

ASH Summary Rich & Multidimensional Sampling is the future Overview of system load Drilldowns into problems Sql Session Wait event Details information on problem resolution Sampling is the future Took clean vision to create - Bit of a leap of faith Less Accurate but more powerful than past Inaccuracies unimportant for the most part Feasible since version 7 You can implement it yourself OEM simplifies mining of ASH data Copyright 2006 Kyle Hailey

Q1 How can you immediately find the top IO consuming SQL statement in the last 60 seconds a. v$active_session_history b. v$sqlstats c. v$sql d. v$sqlarea answer a only others b,c,d - only have cumulative values since database startup Copyright 2006 Kyle Hailey

Q2 Which view can you query directly to get the specific waits that occurred 30 minutes ago: a. v$active_session_history b. v$waitclassmetric c. v$system_event d. v$session_wait_history e. v$waitclassmetric_history answer b - last 60 seconds only others a only c - cumulative info since db startup e - only wait groups, not wait events, but has the history for last hour d - last 10 waits only Chapt 1 ASH 8 ==================================================================== analyzing database performance. The view v$active_session_history is new in 10g, ASH (v$active_session_history) is a revolutionary data source for monitoring and but most of the data needed in order to simulate v$active_session_history by hand has been available since which version b. 7 a. 6 c. 8 e. 10 d. 9 which is the foundation for ASH b - since version 7 when wait events were introduced along with the view v$session_wait 4 ==================================================================== How can immediate find the top IO consuming SQL statement in the last 60 seconds c. v$sql b. v$sqlstats a. v$active_session_history d. v$sqlarea b,c,d - only have cumulative values since database startup a only b - last 60 seconds only c - cumulative info since db startup d - last 10 waits only e - only wait groups, not wait events, but has the history for last hour Copyright 2006 Kyle Hailey

Q3 ASH (v$active_session_history) is a revolutionary data source for monitoring and analyzing database performance. The view v$active_session_history is new in 10g, but most of the data needed in order to simulate v$active_session_history by hand has been available since which version a. 6 b. 7 c. 8 d. 9 e. 10 b - since version 7 when wait events were introduced along with the view v$session_wait which is the foundation for ASH answer Chapt 1 ASH 4 ==================================================================== How can immediate find the top IO consuming SQL statement in the last 60 seconds c. v$sql b. v$sqlstats a. v$active_session_history d. v$sqlarea others a only b,c,d - only have cumulative values since database startup Copyright 2006 Kyle Hailey