A first look at table partitioning PUG Challenge Americas Richard Banville & Havard Danielsen OpenEdge Development June 9, 2014

© 2014 Progress Software Corporation. All rights reserved. 2 The Steps To Table Partitioning 1)Why are you partitioning? 2)Identify tables a)Partitioning Strategy b)See white paper 3)Preparation a)Type II Storage Areas b)Recid/rowid usage c)Aligned field assignments d)Partition aligned (local) indexes 4)Migration / adoption strategy a)Dump / Load b)“In place” migration 5)Current backup 6)Enable partitioning a)Database b)Tables / indexes / lobs 7)Split out existing data 8)Truncate / de-allocate “initial” partition

© 2014 Progress Software Corporation. All rights reserved. 3 Disclaimer  This presentation is for informational purposes only, and the reader is hereby cautioned that actual product development may vary significantly from it  This presentation may not be interpreted as any commitment on behalf of Progress, and future development, timing and release of any features or functionality described in this presentation remains at our sole discretion

© 2014 Progress Software Corporation. All rights reserved. 4 1)Why are you partitioning? Advantages  Performance impact Partition elimination for queries (“pruning”) Improved concurrency –For random activity  Availability  Maintenance advantages Purge, Archive Repair, rebuild, recover Partition level tuning Disadvantages  Partition alignment & lookup (insert / delete) Update of partition aligned key values Missing aligned columns in where clause or record creation  Getting it right the first time Knowledge of application table definition & physical layout Repartitioning costs  More complex deployment

© 2014 Progress Software Corporation. All rights reserved. 5 2)Identify tables & Partitioning strategy  “Stable” data logically grouped by Chronological events (range partitions) Discrete list of values (list partitions) Sub-partition partitioning –Partitioning the same table according to the values of more than one column  Table / index maintenance too high Operational time Data availability  See white paper: //community.progress.com/technicalusers/f/18/t/9294.aspx//community.progress.com/technicalusers/f/18/t/9294.aspx

© 2014 Progress Software Corporation. All rights reserved. 6 2)Identify tables & Partitioning strategy Sub-partitioning Strategy first! 9 partition example Sub-partition by region & order-date w/in region {List, Range} Only last partitioned column may be a range partition Range partition can be any “indexable” field type List / ranges must be inclusive, not necessarily symmetrical Storage considerations for new partitions

© 2014 Progress Software Corporation. All rights reserved. 7 3)Preparation  Type II Storage Areas All associated data (Table, indexes, lobs) Storage considerations  Aligned field assignment Columns in partition definition cannot be UNKNOWN Avoid multiple assign statements Otherwise, use a partition defined default # Storage areas Extent location Cluster size Records per block Toss limits Buffer pool assignment

© 2014 Progress Software Corporation. All rights reserved. 8 3)Preparation  Recid / Rowid usage Recids will NOT work Rowids can change at runtime –Browser’s result sets updated internally  Rowid generation USING OpenEdge.DataAdmin.Util.*. –RowidGenerator: TableStart(table, min, max, pid) –myROWID = RowidGenerator: GetNextRowid()  Where’s my row? BUFFER-PARTITION-ID:getByHdl(handle) BUFFER-PARTITION-ID:getByRowid(rowid) BUFFER-PARTITION-ID(buffer)

© 2014 Progress Software Corporation. All rights reserved. 9 Where’s my row? Find first order NO-LOCK. Find _File where _File-name = “Order” NO-LOCK. Find _StorageObject where _Object-Number = _File-num AND _PartitionId = BUFFER-PARTITION-ID(Order) AND _Object-Type = 1 NO-LOCK. Find _Area of _StorageObject NO-LOCK. Find _Partition-policy-detail where _Partition-policy-detail._Partition-id = BUFFER-PARTITION-ID(Order) AND _Partition-policy-detail._Object-Number = _File-num NO-LOCK. Display _Area-name _File-name _Partition-name. Find first order NO-LOCK. Find _File where _File-name = “Order” NO-LOCK. Find _StorageObject where _Object-Number = _File-num AND _PartitionId = BUFFER-PARTITION-ID(Order) AND _Object-Type = 1 NO-LOCK. Find _Area of _StorageObject NO-LOCK. Find _Partition-policy-detail where _Partition-policy-detail._Partition-id = BUFFER-PARTITION-ID(Order) AND _Partition-policy-detail._Object-Number = _File-num NO-LOCK. Display _Area-name _File-name _Partition-name.

© 2014 Progress Software Corporation. All rights reserved. 10 3)Preparation - Index Support Composite Partitioning Order Table Western Region Northern Region Southern Region 12/31/ /31/ /31/ /31/ partition example Sub-partition by region & order-date w/in region IndexComponentsPartition Aligned Index #1 (local){Region, Order-Date, Name}YES Index #2 (local){Region, Order-Date, S-rep}YES Index #3 (global){Cust-num}NO Global indexes span partitions Indexed field need not be partition aligned Idx #3 Local indexes on order-date, name Idx #1 Local indexes on order-date, S-rep Idx #2 9 B-trees supporting 3 index definitions Local indexes on order-date, name Idx #1 Local indexes on order-date, S-rep Idx #2

© 2014 Progress Software Corporation. All rights reserved. 11 4)Migration/adoption strategy – New tables (or Dump / Load) Sub-partitioning 1.Dump data* 2.Delete table* 3.Add table 4.Define / Add partitions 5.Load data 6.Have a nice day 9 partition example Sub-partition by region & order-date w/in region

© 2014 Progress Software Corporation. All rights reserved. 12 4)Migration/adoption strategy – In place migration Composite Partitioning Order Table Western Region Northern Region Southern Region 06/30/2014 Implemented for migration only Tool creates list of unique entries Can modify range of given list Partition 0 Data logically segregated Single composite partition example Sub-partition by region & order-date w/in region

© 2014 Progress Software Corporation. All rights reserved. 13 4)Migration/adoption strategy – In place migration Composite Partitioning Order Table Western Region Northern Region Southern Region 06/30/2014 Implemented for migration only Tool creates list of unique entries Can modify range of given list Partition 0 Data logically segregated Single composite partition example Sub-partition by region & order-date w/in region

© 2014 Progress Software Corporation. All rights reserved. 14 4)Migration/adoption strategy – In place migration Composite Partitioning Order Table Western Region Northern Region Southern Region 06/30/ /31/2014 Implemented for migration only Tool creates list of unique entries Can modify range of given list Partition 0 Data logically segregated Partitions 1-3 Data physically segregated 4 partition example Sub-partition by region & order-date w/in region

© 2014 Progress Software Corporation. All rights reserved. 15 4)Migration/adoption strategy – In place migration Composite Partitioning Order Table Western Region Northern Region Southern Region 06/30/ /31/2014 Implemented for migration only Tool creates list of unique entries Can modify range of given list 10 Composite Index #10 Partition #0 10 Local Index #10 Partition #1 10 Local Index #10 Partition #2 10 Local Index #10 Partition #3 4 partition example Sub-partition by region & order-date w/in region

© 2014 Progress Software Corporation. All rights reserved. 16 6)Enable partitioning – Database level 5) Have a good backup! proutil -C enabletablepartitioning  _Partition-Policy-Detail (-353) Defines each individual partition  _Partition-Policy (-352) Describes partition at the “table” level

© 2014 Progress Software Corporation. All rights reserved. 17 6)Enable partitioning – Table and indexes Order Table Western Region Northern Region Southern Region 06/30/ Composite Index #10 Partition #0 Migration Single composite partition Composite partition aligned “local” index Sub-partition by region & order-date w/in region Storage considerations for new partitions

© 2014 Progress Software Corporation. All rights reserved. 18 6)Enable partitioning – Table and indexes Order Table Western Region Northern Region Southern Region 06/30/ /31/ Composite Index #10 Partition #0 10 Local Index #10 Partition #1 10 Local Index #10 Partition #2 10 Local Index #10 Partition #3 Migration Composite partition 3 newly added partitions Partition aligned indexed Sub-partition by region & order-date w/in region Storage considerations for new partitions

© 2014 Progress Software Corporation. All rights reserved. 19 7)Split out existing data (optional) Identify created partition as a “split target” Move data into target partitions Transactional scoping by groups of record/index operations –Data for same partition definition spans physical partitions –Only ever one copy of the data Recovery of operation restarts where it left off Online operation with full access to non-split data –New split transitional state for partitions Multiple concurrent operations allowed OpenEdge Replication fully supported proutil -C partitionmanage split table partition | composite "initial“ useindex recs

© 2014 Progress Software Corporation. All rights reserved. 20 7)Split out existing data Order Table Western Region Northern Region Southern Region 06/30/ /31/ Composite Index #10 Partition #0 10 Local Index #10 Partition #1 10 Local Index #10 Partition #2 10 Local Index #10 Partition #3 Migration  Create new partitions  Mark as split target” via OpenEdge Management Composite partition 3 newly added partitions Partition aligned indexed Sub-partition by region & order-date w/in region

© 2014 Progress Software Corporation. All rights reserved. 21 7)Split out existing data Order Table Western Region Northern Region Southern Region 06/30/ /31/ Composite Index #10 Partition #0 10 Local Index #10 Partition #1 10 Local Index #10 Partition #2 10 Local Index #10 Partition #3 Migration 12/31/ Local Index #10 Partition #4 10 Local Index #10 Partition #5 10 Local Index #10 Partition #6  Create new partitions  Mark as split target” via OpenEdge Management Data in this range now has restricted access 3 newly added “split target partitions New partition aligned indexes

© 2014 Progress Software Corporation. All rights reserved Split out existing data Order Table Western Region Northern Region Southern Region 06/30/ /31/ Composite Index #10 Partition #0 10 Local Index #10 Partition #1 10 Local Index #10 Partition #2 10 Local Index #10 Partition #3 Migration proutil -C partitionmanage split table order composite "initial“ recs /31/ Local Index #10 Partition #4 10 Local Index #10 Partition #5 10 Local Index #10 Partition #6

© 2014 Progress Software Corporation. All rights reserved. 23 7)Split out existing data Order Table 12/31/2014 Migration 06/30/ /31/2013 Western Region Northern Region Southern Region proutil -C partitionmanage split table order composite "initial“ recs 1000 Split target 06/30/2014

© 2014 Progress Software Corporation. All rights reserved. 24 proutil -C partitionmanage truncate partition table recs [ deallocate ] 8)Truncate / de-allocate “initial” partition Order Table 12/31/2014 Reclaiming Free Space 06/30/ /31/2013 Western Region Northern Region Southern Region

© 2014 Progress Software Corporation. All rights reserved. 25 proutil -C partitionmanage truncate partition table recs [ deallocate ] 8)Truncate / de-allocate “initial” partition Order Table 12/31/ /30/ /31/2013 Western Region Northern Region Southern Region

© 2014 Progress Software Corporation. All rights reserved. 26 8)Truncate / de-allocate “initial” partition Order Table 12/31/ /30/ /31/2013 Western Region Northern Region Southern Region Reclaiming Free Space

© 2014 Progress Software Corporation. All rights reserved. 27