1. IO Waits Kyle Hailey Kyle_hailey@hotmail.com

2. #.2 Copyright 2006 Kyle Hailey Waits Covered in this Section  db file sequential read  db file scattered read  db file parallel read  direct path read  direct path write  sort segment request  direct path write (lob)  file open  file identify

3. #.3 Copyright 2006 Kyle Hailey IO Waits Data Files DBWR Buffer Cache User1 SGA Library Cache Log Buffer

4. #.4 Copyright 2006 Kyle Hailey Table/Index IO  db file sequential read  db file scatted read  db file parallel read

5. #.5 Copyright 2006 Kyle Hailey db file sequential read : Diag ShadowProcess 1) 1)Search Buffer Cache for the block 2) 2)Fails 3) 3)Reads of disk File Block Buffer Cache

6. #.6 Copyright 2006 Kyle Hailey db file sequential read : Wait  #1 Top Wait  Single Block Read  Index  Data via and Index  Data via Rowid  Rollback  Also dumping headers, rebuilding controlfile  Sequential means  Sequence from an Index  Sequence from a rowid

7. #.7 Copyright 2006 Kyle Hailey db file sequential read : Args  P1 = File #  P2 = Block #  P3 = 1  Can be more than 1 for temporary segments select segment_name, segment_type from dba_extents where file_id = P1 and P2 between block_id and block_id + blocks – 1; select segment_name, segment_type from dba_extents where file_id = P1 and P2 between block_id and block_id + blocks – 1;

8. #.8 Copyright 2006 Kyle Hailey db file scattered read : Diag ShadowProcess 1) 1)Search Buffer Cache for the blocks 2) 2)Fails 3) 3)Reads off disk File Block Multi Block Read Count Buffer Cache

9. #.9 Copyright 2006 Kyle Hailey db file scattered read : Wait  # 3 Top Wait Event  Multi Block Read  Full Table Scan  Index Fast Full Scans  Scattered Means  Blocks are read and scattered throughout buffer cache

10. #.10 Copyright 2006 Kyle Hailey db file scattered read : Args  P1 = File #  P2 = Block #  P3 = # of Blocks to Read select segment_name, segment_type from dba_extents where file_id = P1 and P2 between block_id and block_id + blocks – 1; select segment_name, segment_type from dba_extents where file_id = P1 and P2 between block_id and block_id + blocks – 1;

11. #.11 Copyright 2006 Kyle Hailey db file parallel read : Diag ShadowProcess 1) 1)Search Buffer Cache for the blocks 2) 2)Fails 3) 3)Reads that block off Disk Buffer Cache

12. #.12 Copyright 2006 Kyle Hailey db file parallel read : Wait  #18 Top Wait  Process issues multiple single block reads in parallel  Documentation says only for recovery  But seems to happen for normal read ops as well  Async Call – wait for all reads to complete

13. #.13 Copyright 2006 Kyle Hailey db file parallel read: Args  P1 = # of files to read from  P2 = # of total blocks to read  P3 = # of I/O requests  Same value as P2

14. #.14 Copyright 2006 Kyle Hailey IO Solutions  If  Db scattered Read  Db file sequential Read  Db file parallel Read  Then  Check average read times per file  Should be between 5-15 ms  Data in Statspack under “File IO Stats”  Check Cache buffer Hit ratio  Check db_cache_advice 9i and higher  Data in Statspack under “Buffer Pool Advisory”  Tune High IO SQL

15. #.15 Copyright 2006 Kyle Hailey IO Solutions : Ave Read Time File IO Stats DB/Inst: LABSF03/labsf03 Snaps: 1-2 ->Mx Rd Bkt: Max bucket time for single block read ->ordered by Tablespace, File Tablespace Filename ------------------------ ---------------------------------------------------- Av Mx Av Av Rd Rd Av Av Buffer BufWt Reads Reads/s (ms) Bkt Blks/Rd Writes Writes/s Waits (ms) ---------- ---- ------- ----- --- ------- ------------ -------- ---------- -- SYSTEM /u01/app/oracle/oradata/labsf03/system01.dbf 445 15 0.4 16 1.0 1,157 39 2,744 93.3 USERS /u01/app/oracle/oradata/labsf03/users01.dbf 223 7 0.5 ### 1.0 9,725 324 4 100.0 File IO Stats DB/Inst: LABSF03/labsf03 Snaps: 1-2 ->Mx Rd Bkt: Max bucket time for single block read ->ordered by Tablespace, File Tablespace Filename ------------------------ ---------------------------------------------------- Av Mx Av Av Rd Rd Av Av Buffer BufWt Reads Reads/s (ms) Bkt Blks/Rd Writes Writes/s Waits (ms) ---------- ---- ------- ----- --- ------- ------------ -------- ---------- -- SYSTEM /u01/app/oracle/oradata/labsf03/system01.dbf 445 15 0.4 16 1.0 1,157 39 2,744 93.3 USERS /u01/app/oracle/oradata/labsf03/users01.dbf 223 7 0.5 ### 1.0 9,725 324 4 100.0

16. #.16 Copyright 2006 Kyle Hailey IO Solutions : Buffer Pool Advisory Buffer Pool Advisory Size for Size Buffers for Read Estimated P Est (M) Factor Estimate Factor Physical Reads --- -------- ------ ------------ ------ -------------- D 56.1 6,986 2.3 58,928 D 112.2 13,972 1.6 42,043 D 224.4 27,944 1.0 25,772 D 336.6 41,916 1.0 25,715 D 448.8 55,888 1.0 25,715 D 596 1.0 74,351 1.0 25,715 D 728 1.2 90,818 1.0 25,715 D 840 1.4 104,790 1.0 25,715 D 952 1.6 118,762 1.0 25,715 D 1,064 1.8 132,734 1.0 25,715 Buffer Pool Advisory Size for Size Buffers for Read Estimated P Est (M) Factor Estimate Factor Physical Reads --- -------- ------ ------------ ------ -------------- D 56.1 6,986 2.3 58,928 D 112.2 13,972 1.6 42,043 D 224.4 27,944 1.0 25,772 D 336.6 41,916 1.0 25,715 D 448.8 55,888 1.0 25,715 D 596 1.0 74,351 1.0 25,715 D 728 1.2 90,818 1.0 25,715 D 840 1.4 104,790 1.0 25,715 D 952 1.6 118,762 1.0 25,715 D 1,064 1.8 132,734 1.0 25,715

17. #.17 Copyright 2006 Kyle Hailey IO Solutions  After Checking  File IO response times  Buffer Cache Hit Ratio  Then the only choice is to  Tune the SQL

18. #.18 Copyright 2006 Kyle Hailey IO Throughput

19. #.19 Copyright 2006 Kyle Hailey Direct I/O WAITS Direct I/O : This mechanism lets the client bypass the buffer cache for I/O intensive operations. The disk blocks are written into and read from process private memory.  Direct IO Clients :  hash join  sqlldr  loading/reading LOBs  parallel DMLs  sorts  create table as select  create index

20. #.20 Copyright 2006 Kyle Hailey direct path read  # 7 Top Wait  Could signal sort problems  Sorts read into their PGA from datafiles via direct path read  Other Uses  PQO  Asyncronous read ahead

21. #.21 Copyright 2006 Kyle Hailey direct path read : Diag ShadowProcess Buffer Cache PGA X

22. #.22 Copyright 2006 Kyle Hailey direct path write  # 8 Top Wait  Occurs when:  Could signal sorting waits  Sorts write to TEMP segment via direct path write  insert /*+ APPEND */  sql*loader direct=y

23. #.23 Copyright 2006 Kyle Hailey direct path write : Diag ShadowProcess Buffer Cache PGA TEMP DBWR

24. #.24 Copyright 2006 Kyle Hailey sort segment request  #26  SMON allocates sort space in Temp Tablespace  If SMON is busy this can cause waits for sort space  SMON may be stuck rolling back

25. #.25 Copyright 2006 Kyle Hailey direct path write (lob)  #35  Note: Direct path read (lob)  Happens for reads on LOBs that are stored NOCACHE  LOBS stored CACHE are read into the buffer cache

26. #.26 Copyright 2006 Kyle Hailey file open  #16  Args  P1 fib  P2 iov  P3 0  Wait on an open file  The time it takes to open the file  Times should be order of a block read  To tune, try pre-opening files  Leave sessoins connected  High reconnection rates can exacerbate  Can try holding log files open (see script from Steve Adams)

27. #.27 Copyright 2006 Kyle Hailey file identify  #29  The time it takes to identify a file so that it can be opened later.  Datafile should be identified just once  Logfiles are identified every checkpoint  Reduce checkpoints  Arguments  p1 fib  p2 filename  p3 opcode (1 test for create, 2 get details)

28. #.28 Copyright 2006 Kyle Hailey Summary I/O  direct path read : sorts or PQO  direct path write : direct path or temp io  sort segment request : SMON occupied?  direct path write (lob) : stored NOCACHE  file open : log files or session reconnects  file identify : log files or session reconnects

29. #.29 Copyright 2006 Kyle Hailey Summary I/O  Buffer Cache IO  db file sequential read  db file scattered read  db file parallel read  Tune  I/O’s should be 5-15ms  Check Buffer Cache Advisory  Tune SQL