1. OPS-5: DataServer Best Practices Revised & Revisited David Moloney Software Architect

2. © 2008 Progress Software Corporation2 OPS-5: DataServer Best Practices – Revisited & Revised D I S C L A I M E R Under Development  This talk includes information about potential future products and/or product enhancements.  What I am going to say reflects our current thinking, but the information contained herein is preliminary and subject to change. Any future products we ultimately deliver may be materially different from what is described here. D I S C L A I M E R

3. © 2008 Progress Software Corporation3 OPS-5: DataServer Best Practices – Revisited & Revised Agenda  DataServers & Best Practices The path to successful DataServer development & deployment Why ? When to use ? What are they ? Why ? When to use ? What are they ?

4. © 2008 Progress Software Corporation4 OPS-5: DataServer Best Practices – Revisited & Revised DataServers: WHY ?  Has a customer, your partners or your company ever asked you: “Does your OpenEdge Application run against” MS SQL Server ? Oracle ® ? Some other data source ?

5. © 2008 Progress Software Corporation5 OPS-5: DataServer Best Practices – Revisited & Revised DataServers WHEN ?  When “yes” on the WHY question would be compelling  “Yes” is “compelling” when there’s a willingness to: “Invest” mind share in the foreign data source Re-examine parts of your OpenEdge application Compromise performance for conformance … in light of a “compelling” business case, corporate decision or market opportunity.

6. © 2008 Progress Software Corporation6 OPS-5: DataServer Best Practices – Revisited & Revised What is a DataServer ?  A “redirection” layer for database requests in an OpenEdge client or agent.  OpenEdge database request translation  Interface layer communication with a foreign data source

7. © 2008 Progress Software Corporation7 OPS-5: DataServer Best Practices – Revisited & Revised What is a DataServer ? OpenEdge Database OpenEdge Database

8. © 2008 Progress Software Corporation8 OPS-5: DataServer Best Practices – Revisited & Revised What is a DataServer ? MS SQL Server MS SQL Server INTVARCHAR 3B 4Gates

9. © 2008 Progress Software Corporation9 OPS-5: DataServer Best Practices – Revisited & Revised What is a DataServer ? Oracle SQL Server Oracle SQL Server INTVARCHAR2 5L 6Ellison MS SQL Server MS SQL Server INTVARCHAR 3B 4Gates

10. © 2008 Progress Software Corporation10 OPS-5: DataServer Best Practices – Revisited & Revised What is a DataServer ? Oracle SQL Server Oracle SQL Server INTVARCHAR2 5L 6Ellison MS SQL Server MS SQL Server INTVARCHAR 3B 4Gates

11. © 2008 Progress Software Corporation11 OPS-5: DataServer Best Practices – Revisited & Revised What is a DataServer ? Oracle SQL Server Oracle SQL Server INTVARCHAR2 5L 6Ellison MS SQL Server MS SQL Server INTVARCHAR 3B 4Gates (process)

12. © 2008 Progress Software Corporation12 OPS-5: DataServer Best Practices – Revisited & Revised What is a DataServer ? OpenEdge Database OpenEdge Database OpenEdge Database OpenEdge Database

13. © 2008 Progress Software Corporation13 OPS-5: DataServer Best Practices – Revisited & Revised What is a DataServer ? Oracle SQL Server Oracle SQL Server INTVARCHAR2 5L 6Ellison MS SQL Server MS SQL Server INTVARCHAR 3B 4Gates OpenEdge Database OpenEdge Database INTCHAR

14. © 2008 Progress Software Corporation14 OPS-5: DataServer Best Practices – Revisited & Revised Agenda  DataServers & Best Practices The path to successful DataServer development & deployment Why √ When to use √ What are they √ Why ? When to use ? What are they ?

15. © 2008 Progress Software Corporation15 OPS-5: DataServer Best Practices – Revisited & Revised WHY Use DataServer Best Practices ?  We’re twice as likely to avoid pain than to pursue gain  Why aren’t DataServers just painless “load and go” ?

16. © 2008 Progress Software Corporation16 OPS-5: DataServer Best Practices – Revisited & Revised WHY Use DataServer Best Practices ?  Appreciate: You are replacing the strengths of your OpenEdge Database with those of another data source: Deference is given to the foreign data source on: –Set-based SQL design –Cursor Consistency –Security –Transaction Control –Lock Management –Query Optimization & Indexing –Constraints

17. © 2008 Progress Software Corporation17 OPS-5: DataServer Best Practices – Revisited & Revised WHY Use DataServer Best Practices ?  Appreciate: You are replacing the features of your OpenEdge Database with those of another data source: “Context” resides in the foreign data source for: –Data Types: Fixed versus variable length strings –Sequences, Triggers, Constraint implementations –Schema, Naming, Qualifying, Networking Conventions –NULLs versus UNKNOWNs –PL/SQL or T-SQL versus ABL

18. © 2008 Progress Software Corporation18 OPS-5: DataServer Best Practices – Revisited & Revised WHY Use DataServer Best Practices ?  Because you need a map !

19. © 2008 Progress Software Corporation19 OPS-5: DataServer Best Practices – Revisited & Revised WHEN To Use DataServer Best Practices ?  Always !  The “Flavors” of pain relief: Minimize loss of performance Maximize conformance Reduce time to market

20. © 2008 Progress Software Corporation20 OPS-5: DataServer Best Practices – Revisited & Revised WHEN To Use DataServer Best Practices ?  Conformance: Example Picking an Index FOR EACH customer: END. FOR EACH employee USE-INDEX department: END. FOR EACH invoice BY zip-code: END.

21. © 2008 Progress Software Corporation21 OPS-5: DataServer Best Practices – Revisited & Revised WHEN To Use DataServer Best Practices ?  Performance: Example Who should do the join ? FOR EACH customer, EACH order OF customer QUERY-TUNING (NO-JOIN-BY-SQLDB): DISPLAY customer.name customer.cust-num customer.postal-code order.order-date. END.

22. © 2008 Progress Software Corporation22 OPS-5: DataServer Best Practices – Revisited & Revised WHEN To Use DataServer Best Practices ?  Performance: Example What will be joined ? FOR EACH customer, EACH order OF customer WHERE LENGTH(RIGHT-TRIM(customer.name)) > 5: DISPLAY customer.name customer.cust-num customer.postal-code order.order-date. END.

23. © 2008 Progress Software Corporation23 OPS-5: DataServer Best Practices – Revisited & Revised WHEN To Use DataServer Best Practices ?  Performance: Example Where is the join done ? FOR EACH customer, FIRST order OUTER-JOIN OF customer WHERE order.order-num < 50: DISPLAY customer.name customer.cust-num customer.postal-code order.order-date. END.

24. © 2008 Progress Software Corporation24 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ? “Steps for Successful DataServer Development and Deployment”  http://www.psdn.com/library/entry.jspa?externalID=1320 http://www.psdn.com/library/entry.jspa?externalID=1320 (Located on PSDN) DataServer Guides  Chapter(s) on “Programming Considerations” The Best Practices White Paper

25. © 2008 Progress Software Corporation25 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ? STEP ONE: Eliminate DBMS differences STEP TWO: Optimize the Application, the DataServer, and the DBMS. In the Best Practices White Paper …

26. © 2008 Progress Software Corporation26 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ?  First get it to work Compile incompatibilities are minimal –SETUSERID –COUNT-OF –CONTAINS –RECID ** ** If ROWID index is not a unique integer STEP ONE: Eliminate DBMS differences

27. © 2008 Progress Software Corporation27 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ?  Then there are the nuances … FIND cust WHERE name BEGINS “SI”. FOR EACH cust WHERE name <= CHR(126). FOR EACH cust WHERE name <> “Bob” STEP ONE: Eliminate DBMS differences

28. © 2008 Progress Software Corporation28 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ?  Record Scoping & Availability STEP ONE: Eliminate DBMS differences DEFINE BUFFER xcust FOR cust. CREATE cust. ASSIGN cust-num = 111. FIND xcust WHERE cust.cust-num = 111.

29. © 2008 Progress Software Corporation29 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ?  Record Scoping & Availability STEP ONE: Eliminate DBMS differences DEFINE BUFFER xcust FOR cust. CREATE cust. ASSIGN cust-num = 111. VALIDATE cust. FIND xcust WHERE cust.cust-num = 111.

30. © 2008 Progress Software Corporation30 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ?  Record Scoping & capturing validation STEP ONE: Eliminate DBMS differences FOR FIRST cust EXCLUSIVE-LOCK: ASSIGN name = FILL(“a”,35) NO-ERROR. VALIDATE NO-ERROR. IF error-status:error THEN DO: UNDO, LEAVE. END. END.

31. © 2008 Progress Software Corporation31 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ?  Record Scoping & Transactions STEP ONE: Eliminate DBMS differences DEF VAR num AS INT INIT 103. DO TRANSACTION: FIND cust WHERE cust = num EXCLUSIVE-LOCK. ASSIGN name = “Bob”. END. FIND cust WHERE cust-num = num. DISPLAY name.

32. © 2008 Progress Software Corporation32 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ?  Record Scoping & Transactions STEP ONE: Eliminate DBMS differences DO TRANSACTION: FIND FIRST cust EXCLUSIVE-LOCK. END. RELEASE cust. DO TRANSACTION: FIND FIRST cust EXCLUSIVE-LOCK. END.

33. © 2008 Progress Software Corporation33 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ?  Record Scoping & Transactions STEP ONE: Eliminate DBMS differences REPEAT: FIND FIRST cust. ASSIGN cust-num = 1. do-blk: DO ON ERROR UNDO do-blk, RETRY do-blk: FIND state WHERE st.st = cust.st. SET state. END. END. DISPLAY state.

34. © 2008 Progress Software Corporation34 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ?  Record Scoping & Transactions STEP ONE: Eliminate DBMS differences REPEAT: FIND FIRST cust. ASSIGN cust-num = 1. do-blk: DO ON ERROR UNDO do-blk, RETRY do-blk: FIND state WHERE st.st = cust.st. SET state. VALIDATE state. END. END. DISPLAY state.

35. © 2008 Progress Software Corporation35 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ?  Record Scoping & Visibility STEP ONE: Eliminate DBMS differences FOR FIRST cust NO-LOCK BY cust-num: END. IF AVAILABLE cust THEN MESSAGE “cust found” VIEW-AS ALERT-BOX.

36. © 2008 Progress Software Corporation36 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ?  Record Scoping & Locking STEP ONE: Eliminate DBMS differences FOR FIRST cust: FOR FIRST cust NO-LOCK: FOR FIRST cust SHARE-LOCK: FOR FIRST cust EXCLUSIVE-LOCK:

37. © 2008 Progress Software Corporation37 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ?  Record Scoping & Cursor Positioning STEP ONE: Eliminate DBMS differences FIND cust WHERE cust-num = 123. FIND cust WHERE name = “Bob”.

38. © 2008 Progress Software Corporation38 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ?  Just code around certain nuances STEP ONE: Eliminate DBMS differences &GLOBAL-DEFINE DB-TYPE ORACLE &IF NOT DEFINED ({&MSSQLS}) &THEN … IF DBTYPE(dbname) = “PROGRESS” THEN RUN OpenEdge-database-code ELSE RUN DataServer-foreign-interface-code

39. © 2008 Progress Software Corporation39 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ?  Be explicit about: Scope of your records Timing of your write operations STEP ONE: Eliminate DBMS differences VALIDATE RELEASE RECID

40. © 2008 Progress Software Corporation40 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ?  Code knowing that for DataServers: Transactions narrow record scoping Transactions release locks and should release buffers STEP ONE: Eliminate DBMS differences DO TRANSACTION: ASSIGN cust.name = “Bob”. END. RELEASE cust.

41. © 2008 Progress Software Corporation41 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ?  Avoid SHARE-LOCK  Adopt a “pessimistic“ locking model: –NO-LOCK –EXCLUSIVE-LOCK STEP ONE: Eliminate DBMS differences FOR EACH cust EXCLUSIVE-LOCK: ASSIGN name = “Bob”. END.

42. © 2008 Progress Software Corporation42 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ?  DataServers only reposition FIND statement “cursors”. Set-based result statements are “contained”. FOR EACH QUERY DYNAMIC FIND  RELEASE set-based buffers that should be out of scope after these, STEP ONE: Eliminate DBMS differences FOR EACH cust: DISPLAY cust. END. RELEASE cust.

43. © 2008 Progress Software Corporation43 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ? STEP ONE: Eliminate DBMS differences

44. © 2008 Progress Software Corporation44 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ? Consider the difference STEP ONE: Eliminate DBMS differences FIND customer. versus FOR EACH customer.

45. © 2008 Progress Software Corporation45 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ? Consider the difference STEP ONE: Eliminate DBMS differences FIND customer USE-INDEX cust-num. versus FOR EACH customer BY cust-num.

46. © 2008 Progress Software Corporation46 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ?  Code with a set-based mind set ! FOR EACH QUERY DYNAMIC FIND INDEXED-REPOSITION  Avoid OLTP-biased FIND statements STEP ONE: Eliminate DBMS differences

47. © 2008 Progress Software Corporation47 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ? STEP ONE: Eliminate DBMS differences

48. © 2008 Progress Software Corporation48 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ? STEP TWO: Optimize the Application, the DataServer, and the DBMS. On PSDN, see “Performance Study” at the end of “DB-16: In Any Case, the Devil’s in the DataServer Details”: http://www.psdn.com/library/entry.jspa?externalID=3355 STEP ONE: Eliminate DBMS differences

49. © 2008 Progress Software Corporation49 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ?  ROWID optimizes OpenEdge database access  DataServers map ROWID to a unique key  Uniqueness in DataServers enables: Locking Deletions & Updates FINDs, Queries & Browsers All cursor positioning such as INDEXED-REPOSITION RECID/ROWID functions random record access in a non-unique set STEP TWO: Optimize Application, DataServer, and DBMS.

50. © 2008 Progress Software Corporation50 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ?  Set-based design enhances ROWID reconciliation FOR EACH QUERY DYNAMIC FIND INDEXED-REPOSITION STEP TWO: Optimize Application, DataServer, and DBMS.

51. © 2008 Progress Software Corporation51 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ? #1. Eliminate FIND statements FIND FIRST cust FOR FIRST cust:END. FIND LAST cust FOR LAST cust:END. STEP TWO: Optimize Application, DataServer, and DBMS.

52. © 2008 Progress Software Corporation52 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ? FIND FIRST Customer NO-LOCK NO-ERROR. IF AVAILABLE Customer THEN Cnt = 1. REPEAT: FIND NEXT Customer NO-ERROR. IF NOT AVAILABLE (Customer) THEN LEAVE. Cnt = Cnt + 1. END. OPEN QUERY q FOR EACH Customer NO-LOCK. REPEAT: GET NEXT q. IF NOT AVAILABLE Customer THEN LEAVE. Cnt = Cnt + 1. END. CLOSE QUERY q. STEP TWO: Optimize Application, DataServer, and DBMS.

53. © 2008 Progress Software Corporation53 OPS-5: DataServer Best Practices – Revisited & Revised WHAT are the DataServer Best Practices ? #2: Stored Procedures #3: Avoid SHARE-LOCK, Consider NO-LOCK #4: Tune your database, especially indexes #5: Resolve queries/joins on the server #6: Use field lists and/or views #7: Use QUERY-TUNING where appropriate #8: Use –Dsrv connect options #9: Client startup options STEP TWO: Optimize Application, DataServer, and DBMS.

54. © 2008 Progress Software Corporation54 OPS-5: DataServer Best Practices – Revisited & Revised In Summary #10. STEP ONE: Eliminate DBMS differences STEP TWO: Optimize Application, DataServer, and DBMS. STEP ONE resonates with STEP TWO

55. © 2008 Progress Software Corporation55 OPS-5: DataServer Best Practices – Revisited & Revised For More Information, go to… Basic Tools:  Manual Refactoring (and 10.1C unsupported switch “-clw”) ???  Performance logs  Progress ® Debugger  Compile XREF  Progress Profiler Tool (unsupported but ships w/OpenEdge) Other Resources:  Proparse: http://www.joanju.comhttp://www.joanju.com  Prolint-http: http://www.prolint.orghttp://www.prolint.org  ProRefactor: http://www.prorefactor.orghttp://www.prorefactor.org  PSDN OpenEdge DataServers category (Best Practices paper, Performance Study)OpenEdge DataServers category Forum: OpenEdge RDBMS & Data Management Redevelopment Tools

56. © 2008 Progress Software Corporation56 OPS-5: DataServer Best Practices – Revisited & Revised Relevant Exchange Sessions  OPS-25: Unicode & the DataServer Wednesday, June 11 th 1:30 – 2:30 Grand Ballroom L-M

57. © 2008 Progress Software Corporation57 OPS-5: DataServer Best Practices – Revisited & Revised Questions ?

58. © 2008 Progress Software Corporation58 OPS-5: DataServer Best Practices – Revisited & Revised Thank You

59. © 2008 Progress Software Corporation59 OPS-5: DataServer Best Practices – Revisited & Revised