DB-01 Upgrading to OpenEdge ® Practices & Initial Tuning Tom Harris, Managing Director, RDBMS Technology

2© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Audience Survey n V7-V8.1 Users ? n V8.2 or 8.3 Users? n V9 Users? n OpenEdge 10 Migration done? n OpenEdge 10 Migration in the next 6 months?

3© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Why Are You Going to OpenEdge 10? n Start building new Integratable applications? n Performance ? n Data Types ? n New Language Features (JTA, ProDataSet)? n High Availability (online operations) n Large Databases ? “yeah, we do that...”

4© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Agenda n Conceptual Differences V8-V9-OE10 n The “5-minute rule” n What happens during the 5-minutes ? n Staging A Move n Moved – What Tuning Should I Know About? n Questions for the Developers

5© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Unreleased Feature InformationCaution: Features labeled “10.1A” are presently in development and may possibly encounter problems that prevent their availability in the release. Features labeled “10.1A” are presently in development and may possibly encounter problems that prevent their availability in the release. 10.1A work is in red. If red doesn’t work for you, we’ll also underline it. 10.1A work is in red. If red doesn’t work for you, we’ll also underline it. Until they are available – they are “futures”!

6© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices D I S C L A I M E R Under Development n This talk includes information about potential future products and/or product enhancements. n What I am going to say reflects our current thinking, but the information contained herein is preliminary and subject to change. Any future products we ultimately deliver may be materially different from what is described here. D I S C L A I M E R

7© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Conceptual Differences ReleaseContentionDataIndex Schema Chgs V7 V8 V9 OE10 DB levelMixed Synchronous Off-Line DB levelMixedOff-Line Area levelby areaOn-Line Object levelcluster Optimistic On-Line Schema Basic 4GL +SQL Schema +audit schema in 10.1A Basic 4GL Synchronous

8© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Conceptual Differences (cont.) ReleaseCLOBS V7 V8 V9 OE10 Add Extents NOOff-Line NO SQL BLOBS YES NO YES Off-Line ON-Line (10.1A) >2GB files NO YES 9.1D NO Ent.DB Datetime NO SQL SQL, 4GL

9© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Conceptual Differences (cont.) ReleaseThreads V7 V8 V9 OE10 NONE SQL-92, dbtool SQL + utilities NONE Replication triggers, ai logs triggers, logs, Fathom Repl utility 10.1A * AI Mgt FailOver Cluster user script Integrated Ent.DB triggers, ai logs triggers, logs, Fathom Repl user script Integrated Ent.DB user script

10© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Hey! I just need to get over to 10 ! n So... let’s take the fast lane

11© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices The 5-Minute Rule n CONV runs in-place, in 5-minutes n Mostly, we’re upgrading the schema n No changes to record headers n No changes to physical structures n “Before-and-After Backups are good”

12© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices What Happens after 5-minutes? n You can run the database ! n It will very likely run better than before n Batch jobs with big temp-tables get faster n Fragmentation still exists n Most of your DBA scripts should work fine n Optimize when you can...

13© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices The Alternative What if I want more performance than I get with the fast convert ?

14© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices You could plan to dump and load... n Put your schema in its own area n Move to Type II Areas –Group some tables n Similar recordsize n Similar update rates n Similar access modes –Some tables need own area n High create/delete n Very Big Tables n Future focus is on Type II Areas...

15© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Block Sizes n bi Cluster size (1024 – 4096) n bi & ai need to match (8K-16K) n bi file is sequential access – put it by itself n Data Cluster Sizes –Table areas: 64 –Index areas: 8 n Records per block –block size / mean record size –Or 128 and forget it! –1 for index only areas

16© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Staging A Move n Backups – still your friend! n Keep dbanyls & promon data for reference n The usual rules still apply for AI & BI disks n Ditto, for SAN setups n Move to 10 (convert or d/l) n Run UPDATE STATISTICS for SQL n Get that backup n Keep sets of promon scripts as you go

17© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Ready to Optimize Further? Whether You CONV or Dump & Load, there are more gains to be had

18© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Plan for “full production” Database Useful Parameters: n -spin: (5,000 thru 10,000) * # CPUs n -B: As big as you can without paging n -hash: -B/4 when –B > 400,000 n -bibufs: 25 n -Bp avoids impact on production n -omsize select count(*) from _storageObject

19© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices n BIW/AIW –Performs writes to bi/ai –Decreases empty buffer waits n APW –Decreases LRU writes –Reduces checkpoint time –Performs DB buffer pool I/O –Increase until 0 blocks flushed at checkpoint –Decrease if partial bi writes increase –Typically need more if running Direct with I/O n Watch Dog –Monitors connections Plan for “full production” Database

20© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Buffers: -Bt n Temp Tables use Type II Storage Areas –Fast empty/drop –Use “empty temp-table ” –Mitigate space overhead with –Bt –-tmpbsize 1 or -tmpbsize 8 n Records per block n Create/toss limit n Possible wasted space n Additional block format overhead

21© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Index Rebuild n Index Rebuild by area, table or schema –proutil -C idxbuild table n Typical Uses: –Complete D&L process –Bring deactivated indexes online n Online index activate coming soon –Fix corruption or Index reorg n Always sort when asked (if you have the temp space)

22© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Idxbuild – Sort Groups n # of sort groups n -SG: 8 thru 64 – Default: 48 n A larger -SG performs less sort/merge(s) n Databases with many indexes per area use larger SG values n Other values –-TM 32 (merge buffers) –-TB 24 (temp block size) –-T specify a different disk

23© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Index Compaction n proutil -C idxcompact [ n ] n Can be run online n When to run –%utilization 2 –Suggest value of 80% to 90% –100% may cause expensive index block splits

24© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Dump and Load n Dump and Load Indicators –Scatter factor > 4 –Fragmentation Factor > 2 n proutil -C dump. –index 0 –Dump performance over data order –Typically choose index based on read order n proutil -C load.bd build –Load with build indexes –Load to truncated area n (truncate rather than “emptied”)

25© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Promon Scripts n Use promon and sar or perfmon –I/O rates, CPU usage and system paging –Buffers flushed at checkpoint –Time between checkpoints –Buffer Hit ratio –Contention (buffers, locks, latches)

26© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices 10.1A Among new 10.1A DB capabilities... n Logfile scanners: format is changing –scan for error number – still works OK –scan for message text – still works OK n Monitor & add extents on-line if needed n Capture important DB events for analysis –you can enable this capability –it’s useful for operations review/support n Default Enterprise install performs better

27© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices 10.1A: Selective Events – logged to table startup info shutdown info abnormal ends locktable ovfl bigrow bulkload roll forward force options dbrpr dbtool probkup procopy resolve limbo tx prorest prostrct (most) 2pc bgn/end 2pc mod/rcvr protrace files proutil -EnableKeyEvents

28© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Default Parameters: 10.1 A changes –aibufs number of after image buffers –B blocks in the database buffer pool -bibufs before image buffers -G seconds before bi cluster reuse -L number of locktable entries -Ma # remote users per database server -Mm max size of TCP/IP message fragment

29© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Default Parameters in 10.1A (cont.) -semsets number of sets avl. to broker -spin latch tries before pausing DB block size bi cluster size (on-disk block size for notes) bi block I/O block size writing to bi file ai block I/O block size writing to ai file

30© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Summary n There are important changes in the RDBMS n You can get over to OpenEdge10 easily –There is immediate benefit –There is also a lot more to gain! n It’s easy to stage a move in several steps n Have a plan for “full production” n Check promon/SAR as you take each step n Until we ship it, it’s a Future

31© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Questions? Storage Engine Crew

32© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices Thank you for your time! Storage Engine Crew

33© 2005 Progress Software Corporation DB-01 Upgrading to OpenEdge® Practices