1. 1 juliandyke.com © 2005 Julian Dyke Data Segment Compression Julian Dyke Independent Consultant Web Version

2. juliandyke.com © 2005 Julian Dyke 2 Agenda 1.Introduction 2.What is data compression? 3.Data segment compression 1.Functionality 2.Syntax 3.Implementation 4.Performance 4.Conclusion

3. juliandyke.com © 2005 Julian Dyke 3 What is data compression?  Data is  compressed when it is written to a block  decompressed when it is read from the block  Compression requires  less data storage to hold the compressed data  more CPU to compress and decompress the data

4. juliandyke.com © 2005 Julian Dyke 4 Why use data compression?  Data compression  Increases number of rows in each block  Reduces number of blocks required to store data  For a full table scan reduces number of logical (and probably physical) I/Os required  For a table access by ROWID increases probability that block is already in buffer cache  Improves buffer cache hit ratio  Potentially reduces backup and recovery times

5. juliandyke.com © 2005 Julian Dyke 5 When does Oracle use compression?  Oracle compresses some data types including  VARCHAR2  NUMBER  RAW  Oracle does not compress  DATE  CHAR  Compression is often achieved by  using length byte(s)  trimming unused characters/bytes

6. juliandyke.com © 2005 Julian Dyke 6 When does Oracle use compression?  Oracle also compresses  Length bytes in table blocks  Length bytes in index blocks  NULL values  NULL values at end of each row  Index branch blocks (suffix compressed)  Index leaf blocks (optionally prefix compressed)  In addition some data structures implicitly compress data  IOTs  Index Clusters

7. juliandyke.com © 2005 Julian Dyke 7 Data Segment Compression  Introduced in Oracle 9.2  Intended for  DSS environments  Read-only tables  Not intended for  OLTP environments  Environments with any DML activity subsequent to data loading  Data is compressed at block level  Direct path load must be used

8. juliandyke.com © 2005 Julian Dyke 8 Restrictions  Data segment compression cannot be used with  IOTs  IOT overflow segments  IOT mapping tables  Index clustered tables  Hash clustered tables  Hash partitions  Hash / list subpartitions  External Tables

9. juliandyke.com © 2005 Julian Dyke 9 Block Level Compression  Compression is applied at block level  Blocks will only be compressed if  data is sufficiently large to fill the block  rows have low enough cardinality  Columns will be reordered within each block to achieve optimal compression ratios  A segment may contain  compressed and uncompressed blocks  blocks compressed on different columns

10. juliandyke.com © 2005 Julian Dyke 10 Direct Path Load  Direct path load bypasses much on the work done by conventional load  Direct path load  reserves extents from above HWM  formats rows into blocks  inserts blocks back into table  adjusts HWM  No other transactions can be active on the table whilst load is in progress

11. juliandyke.com © 2005 Julian Dyke 11 Direct Path Load  In Oracle 9.2 the following statements can use direct path loads  CREATE TABLE AS SELECT  INSERT /* + APPEND */  ALTER TABLE MOVE  In addition the following features can use direct path loads  Materialized View Refresh  SQL*Loader  Online reorganisation

12. juliandyke.com © 2005 Julian Dyke 12 Creating New Tables  Tables are compressed using COMPRESS clause CREATE TABLE t1 ( c01 NUMBER, c02 VARCHAR2(30) ) COMPRESS;  Default is for tables to be uncompressed  This is equivalent to using the NOCOMPRESS clause

13. juliandyke.com © 2005 Julian Dyke 13 Successful Compression  If conditions are met then these statements create compressed data blocks CREATE TABLE t2 COMPRESS AS SELECT * FROM t1; CREATE TABLE t2 COMPRESS AS SELECT * FROM t1 WHERE ROWNUM < 1; INSERT /*+ APPEND */ INTO t2 SELECT * FROM t1;

14. juliandyke.com © 2005 Julian Dyke 14 Unsuccessful Compression  These statements will not create compressed data blocks CREATE TABLE t2 AS SELECT * FROM t1 WHERE ROWNUM < 1; INSERT INTO t2 VALUES ('DBA_TABLES',46); CREATE TABLE t2 COMPRESS AS SELECT * FROM t1 WHERE ROWNUM < 1; INSERT INTO t2 SELECT * FROM t1;

15. juliandyke.com © 2005 Julian Dyke 15 Altering Existing Tables  Compression can be specified for an existing table ALTER TABLE t1 COMPRESS;  Existing blocks are not compressed  New blocks will be compressed if direct path load is used  Similarly ALTER TABLE t1 NOCOMPRESS;  disables compression for new data blocks, but does not change existing data blocks

16. juliandyke.com © 2005 Julian Dyke 16 Moving Existing Tables  Tables can be moved using ALTER TABLE t1 MOVE COMPRESS;  This command  Creates a new segment  Uses direct load to copy and compress blocks  Drops old segment

17. juliandyke.com © 2005 Julian Dyke 17 Data Dictionary Views  Not modified in Oracle 9.2  Data segment compression flag is missing from  ALL_TABLES  DBA_TABLES  USER_TABLES  Data segment compression is recorded by setting a bit in TAB$.SPARE1  Affects Håkan factor – maximum number of rows that can be held on block

18. juliandyke.com © 2005 Julian Dyke 18 Data Dictionary Views  In Oracle 9.2.0.1 the following will list all compressed tables in the database SELECT u.name AS owner, o.name AS table_name FROM sys.tab$ t, sys.obj$ o, sys.user$ u WHERE BITAND (t.spare1, 131072) = 131072 AND o.obj# = t.obj# AND o.owner# = u.user#;

19. juliandyke.com © 2005 Julian Dyke 19 Tablespace Defaults  Data segment compression can be specified at tablespace level CREATE TABLESPACE ts01 DEFAULT COMPRESS;  All new objects created will have compression enabled  Data segment compression can also be specified for existing tablespaces ALTER TABLESPACE ts01 DEFAULT COMPRESS;

20. juliandyke.com © 2005 Julian Dyke 20 Data Dictionary Views  In Oracle 9.2.0.1 the DBA_TABLESPACES view was not been updated to include data segment compression  Use the following to identify tablespaces with compression enabled SELECT name FROM sys.ts$ WHERE BITAND (flags, 64) = 64;

21. juliandyke.com © 2005 Julian Dyke 21 Partitioned Tables  Data segment compression can also be used with range or list partitioned tables CREATE TABLE t1 (c01 NUMBER, c02 VARCHAR2(200)) PARTITION BY RANGE (c01) ( PARTITION p1 VALUES LESS THAN (100), PARTITION p2 VALUES LESS THAN (200) ) COMPRESS;  Oracle 9.2 cannot compress hash or composite partitioned tables

22. juliandyke.com © 2005 Julian Dyke 22 Partitioned Tables  Can also create a table with some partitions compressed and others uncompressed CREATE TABLE t1 (c01 NUMBER, c02 VARCHAR2(200)) PARTITION BY RANGE (c01) ( PARTITION p1 VALUES LESS THAN (100) COMPRESS, PARTITION p2 VALUES LESS THAN (200) ) COMPRESS; ALTER TABLE t1 ADD PARTITION p3 VALUES LESS THAN (300) COMPRESS;  Compression can also be specified for new partitions added to an existing table

23. juliandyke.com © 2005 Julian Dyke 23 Partitioned Tables  Existing partitions can be specified as compressed/uncompressed using ALTER TABLE t1 MODIFY PARTITION p1 COMPRESS; ALTER TABLE t1 MODIFY PARTITION p1 NOCOMPRESS;  These do not affect existing rows ALTER TABLE t1 MOVE PARTITION p1 COMPRESS;  This  creates new segment,  copies and compresses all the rows  drops old segment  An existing uncompressed partition can be compressed using

24. juliandyke.com © 2005 Julian Dyke 24 Data Dictionary Views  DBA_PART_TABLES.DEF_COMPRESSION contains NONECompression never enabled ENABLEDCompression enabled at table level DISABLEDCompression has been enabled at table level and subsequently disabled N/APartitioned IOT  DBA_TAB_PARTITIONS.COMPRESSION contains ENABLEDCompressed enabled for partition DISABLEDOtherwise

25. juliandyke.com © 2005 Julian Dyke 25 Nested Tables  Compression can be specified for storage table of a nested table CREATE TABLE t1 (c1 NUMBER, c2 TY2) NESTED TABLE c2 STORE AS t2 COMPRESS;  In Oracle 9.2. DBA_NESTED_TABLES has not been updated to indicate that the storage table has been compressed

26. juliandyke.com © 2005 Julian Dyke 26 Materialized Views  Compression can be specified for materialized views CREATE MATERIALIZED VIEW mv1 COMPRESS BUILD IMMEDIATE ENABLE QUERY REWRITE AS SELECT c1, c2, SUM (c3) FROM t1 GROUP BY c1, c2;

27. juliandyke.com © 2005 Julian Dyke 27 Materialized Views  Compression can be also specified for existing materialized views ALTER MATERIALIZED VIEW mv1 COMPRESS;  Data will be compressed the next time the materialized view is refreshed  e.g EXECUTE dbms_mview.refresh ('MV1');

28. juliandyke.com © 2005 Julian Dyke 28 SQL*Loader  SQL*Loader can create data segment compressed blocks using direct path loads  Specified using the parameter DIRECT = TRUE  Conventional loads using SQLLDR do not generate compressed blocks

29. juliandyke.com © 2005 Julian Dyke 29 PCTFREE  Default value of PCTFREE for compressed tables is 0  Can be overridden manually e.g. CREATE TABLE t1 (c01 NUMBER) COMPRESS PCTFREE 10;  In general the default behaviour is preferable

30. juliandyke.com © 2005 Julian Dyke 30 Compression Ratios  Compression ratios vary with  number of rows  number of columns  cardinality of rows  Compression ratios can be improved by sorting table on low cardinality columns prior to loading  Can also be improved by using larger block sizes

31. juliandyke.com © 2005 Julian Dyke 31 Compression Ratios  For example – loading SALES table from sales history demo schema $ORACLE_HOME/demo/schema/sales_history Block Size Uncompressed Size (Blocks) Compressed Size (Blocks) Ratio % 2048187771343371.5 40968983610668.0 81924398285064.7 163842179135362.0  Table contains 1016271 rows

32. juliandyke.com © 2005 Julian Dyke 32 Implementation  Each compressed block contains two tables  Symbol table – contains one row for each individual column value or set of column values  Row table – one row for each row in block  Each column in row table can be a  reference to the symbol table if column is compressed  column value if column is not compressed  Compression is performed at block-level only – no inter-block references

33. juliandyke.com © 2005 Julian Dyke 33 Data Segment Compression Block Common Header Compressed Data Header Row directory Free Space Symbol Table Table directory Data Header Transaction Header Row Data Tail 20 bytes 24 bytes + 24 bytes per ITL entry 14 bytes 16 bytes (variable) 8 bytes 2 bytes per row 4 bytes

34. juliandyke.com © 2005 Julian Dyke 34 Compressed Block Header  Compressed blocks include an extra header.  Header length is variable  Depends on  number of columns  order in which they are compressed  Example of header from block dump r0_9ir2=0x0 mec_kdbh9ir2=0x1 r1_9ir2=0x0 76543210 flag_9ir2=------OC fcls_9ir2[5]={ 0 32768 32768 32768 32768 } perm_9ir2[5]={ 0 1 4 2 3 }

35. juliandyke.com © 2005 Julian Dyke 35 Length Bytes  Column length bytes behave differently in compressed blocks <= 200reference (single column values) column count (multi-column values) > 200 AND < 250length is value - 200 250 (0xFA)length is contained in next two bytes 251 (0xFB)reference is contained in next two bytes

36. juliandyke.com © 2005 Julian Dyke 36 Length Bytes  Examples Byte(s) - HexBytes (s) – DecimalValue C92011 CA2022 CB2033 CC2044 F824848 F924949 FA 00 32250 00 5050 FA 00 33250 00 5151 FA 0F 9F250 15 1593999 FA 0F A0250 15 1604000

37. juliandyke.com © 2005 Julian Dyke 37 Example  Monaco Grand Prix Winners 1993-2002 YearDriverTeam 1993Ayrton SennaMcLaren 1994Michael SchumacherBenetton 1995Michael SchumacherBenetton 1996Olivier PanisLigier 1997Michael SchumacherFerrari 1998Mika HakkinenMcLaren 1999Michael SchumacherFerrari 2000David CoulthardMcLaren 2001Michael SchumacherFerrari 2002David CoulthardMcLaren

38. juliandyke.com © 2005 Julian Dyke 38 Example - Uncompressed Data Block 2001 Ferrari Michael Schumacher 2002 David Coulthard McLaren 1999Ferrari Michael Schumacher 1997 Ferrari Michael Schumacher 1998 Mika Hakkinen McLaren 1993 Ayrton SennaMcLaren 1994Benetton Michael Schumacher 1996Olivier PanisLigier 1995 Michael Schumacher Benetton 2000 David Coulthard McLaren Row Data

39. juliandyke.com © 2005 Julian Dyke 39 Example - Compressed Data Block Row Data Symbol Table 1993 Ayrton Senna4 1996Olivier PanisLigier 199432 1995 32 1997 12 200240 200112 2000 40 1999 12 1998 4Mika Hakkinen Symbol Table Row Data 2 David Coulthard 0 3 Ferrari 1 5 Michael Schumacher 2 Benetton 3 2 4 4 McLaren

40. juliandyke.com © 2005 Julian Dyke 40 Performance  Performance tests  Cost of inserting compressed rows  Cost of selecting compressed rows  Tested using  Oracle 9.2.0.1  Sun Ultra Enterprise 450 – 4 CPUs  8192 byte block size  Test data adapted from sales history demo $ORACLE_HOME/demo/schema/sales_history  SALES table contains 1016271 rows

41. juliandyke.com © 2005 Julian Dyke 41 Inserting Compressed Rows  Loading the entire file into an empty table CompressedBlocksElapsed TIme (Secs) CPU Time (Secs) No439831.774.13 Yes285071.0843.86  Compressed data is 35% smaller  Reduction in logical and physical I/O more than offset by increase in CPU time to compress blocks  Statistics from V$SYSSTAT

42. juliandyke.com © 2005 Julian Dyke 42 Selecting Compressed Rows  Selecting all rows from table CompressedBlocksElapsed TIme (Secs) CPU Time (Secs) No43983.412.77 Yes28503.783.53  Reduction in logical and physical I/O more than offset by increase in CPU time to compress blocks  Statistics from trace file SELECT SUM (quantity_sold) FROM sales;

43. juliandyke.com © 2005 Julian Dyke 43 Caveat  Updating rows is VERY expensive  Rows must be decompressed before they can be updated  In this example the ALL_OBJECTS view contained 7253 rows CREATE TABLE t1 PCTFREE 0 AS SELECT owner, object_name, subobject_name, object_id FROM all_objects;  creates a 28 block table  The same statement with a COMPRESS clause creates a 23 block table

44. juliandyke.com © 2005 Julian Dyke 44 Caveat  The statement UPDATE t2 SET owner = owner;  After the update statement is executed the table contains 79 blocks  Once blocks are decompressed, rollback will not recompress them  Use read-only tablespaces to prevent inadvertent updates  decompresses all blocks  Deletes do not display this characteristic

45. juliandyke.com © 2005 Julian Dyke 45 Conclusions  DSS / read only feature  Good compression ratios  Only works with direct path load  High CPU usage  High elapsed times  Updates are disproportionately expensive  Documentation is weak  Data dictionary views need enhancing

46. juliandyke.com © 2005 Julian Dyke 46 Thank you for your interest For more information and to provide feedback please contact me My e-mail address is: info@juliandyke.com My website address is: www.juliandyke.com