392-"JOIN" us on DataServers Journey June 6th Manchester, NH KLV Prasad – Principal QA Engineer Progress Software vprasad@progress.com Progress

Agenda DataServers Overview DataServers - JOINs Journey of Server-side JOINs in DataServers JOINs Best Practices JOINs Query Performance Tips Future Enhancements Progress

Schema Holder Database DataServers Overview Customer1.. select * from customer ABL Client ABL Client For Each Customer… For Each Customer… ODBC DataServer SQL OpenEdge Database OLTP OLTP OCI MS SQL Server/ Oracle Schema Holder Database Foreign Data Source Progress

DataServers - JOINs Progress

DataServers OpenEdge JOINs – Evolution Join by server Join by client Nested query OF phrase DataServers Join by server Progress

Join by server – How it helps in performance Reduce the number of network turnarounds Optimize the record matching Reduce number of records transferred to the client Progress

DataServers OpenEdge JOINs – Evolution -nojoinbysqldb Join by client Nested query OF phrase DataServers Join by server (Default) Static & Dynamic queries Inner Join Outer Join Single-shot -nojoinbysqldb Progress

Journey of Server-side JOINs in DataServers Progress

Server-side JOINs Journey – By OpenEdge(OE) Releases Inner Join (OE 9.x) Outer Join (OE 11.0) Dynamic Query (OE 11.2) Single-shot (OE 11.3) Multiple Enhancements (OE 11.7) Progress

Left Outer Join – Pre OpenEdge 11.0 DEFINE QUERY q FOR customer, order. OPEN QUERY q FOR EACH CUSTOMER, EACH ORDER WHERE CUSTOMER.CUSTNUM=ORDER.CUSTNUM LEFT OUTER-JOIN. REPEAT: GET NEXT q. IF NOT AVAILABLE customer THEN LEAVE. DISP CUSTOMER.CUSTNUM ORDER.ORDERNUM. END. 1117 records 3953 records SELECT /*+ INDEX_ASC(T0 CUSTOMER##CUSTNUM)*/ * FROM CUSTOMER T0 SELECT /*+ INDEX_ASC(T0 ORDER_##CUSTORDER)*/ * FROM ORDER_ T0 WHERE (CUSTNUM = :1) Client side processing 1117*3953 3953 rows Progress

Left Outer Join – OpenEdge 11.0 Join by client 3953 records 1117 records SELECT /*+ INDEX_ASC(T0 CUSTOMER##CUSTNUM) */ * FROM CUSTOMER T0 SELECT /*+ INDEX_ASC(T0 ORDER_##CUSTORDER) */ * FROM ORDER_ T0 WHERE (CUSTNUM = :1) Client side processing 1117*3953 Supported in Oracle and MSS DataServer Cost based optimizer of foreign Database is utilized in making query access plan OUTER-JOIN request is served by USE_MERGE hint Smaller result set is sent to client for a single database request Handling NULL Joins -JOIN-ON-SELECT, SELECT-ON-JOIN 3953 rows Join by server 3953 records SELECT /*+ USE_MERGE (T0,T1) */ T0.*, T1.* FROM CUSTOMER T0 LEFT OUTER JOIN ORDER_ T1 ON (T1.CUSTNUM = T0.CUSTNUM) Client result set 3953 rows Progress

Server-side Join for Dynamic Queries - OpenEdge 11.2 Dynamic query Server-side Join for Dynamic Queries - OpenEdge 11.2 hQry:QUERY-PREPARE("FOR EACH CUSTOMER , " + "EACH ORDER outer-join of customer"). hQry:QUERY-OPEN(). REPEAT WHILE NOT hQry:QUERY-OFF-END. hQry:GET-NEXT(). IF NOT AVAILABLE customer THEN LEAVE. DISPLAY Customer.CustNum Customer.NAME Order.OrderNum. END. Static queries are by default Join by server Implemented in OpenEdge 11.2 for Oracle DataServer and MSS DataServer Supports all types of joins (inner, outer and self) 3953 records SELECT /*+ USE_MERGE (T0,T1) */ T0.*, T1.* FROM CUSTOMER T0 LEFT OUTER JOIN ORDER_ T1 ON (T1.CUSTNUM = T0.CUSTNUM) Client result set 3953 rows Progress

Server-side single-shot – Pre OpenEdge 11.3 FOR EACH customer, FIRST order WHERE customer.custnum = order.custnum: DISP customer.custnum order.ordernum. END. 1117 records 3953 records SELECT /*+ INDEX_ASC(T0 CUSTOMER##CUSTNUM) */ * FROM CUSTOMER T0 SELECT /*+ INDEX_ASC(T0 ORDER_##CUSTORDER) FIRST_ROWS */ * FROM ORDER_ T0 WHERE (CUSTNUM = :1) ORDER BY CUSTNUM, ORDERNUM Client process 1117 rows Progress

Server-side single-shot – OpenEdge 11.3 Supported only Oracle DataServer Correlated subquery support added in OpenEdge 11.4 Query performance is good if records are evenly distributed in child table for parent table key Join by client is efficient only if child table has tens of thousand matching records for a few parent records Limitations Date Field in WHERE clause Program variable used in WHERE clause BY clause Multiple FIRST/LAST options in the query 1117 records 3953 records SELECT /*+ INDEX_ASC(T0 CUSTOMER##CUSTNUM) */ * FROM CUSTOMER T0 SELECT /*+ INDEX_ASC(T0 ORDER_##CUSTORDER) FIRST_ROWS */ * FROM ORDER_ T0 WHERE (CUSTNUM = :1) ORDER BY CUSTNUM, ORDERNUM Join by client Client process 1117 rows Join by server SELECT customer*,Order* FROM Customer T0, Order T1 WHERE Customer.CustNum = Order.CustNum AND Order.OrderNum = (SELECT MIN (T1.OrderNum) FROM Order WHERE Order.CustNum = Customer.CustNum) 1117 records Client result set 1117 rows Progress

Statement level lock Enhancements- OpenEdge 11.7 MSS DataServer Statement level lock Enhancements- OpenEdge 11.7 Join by server for MSS DataServer (OE 11.7) Lock at record level with benefit of NO-LOCK array fetch Query open with NO-LOCK,SHARE-LOCK – Join by server Query open with EXCLUSIVE-LOCK – Join by client Performance switches QUERY-TUNING (KEYCACHE-JOIN),(NO-KEYCACHE-JOIN) -Dsrv PRGRS_JOIN_NOKEYCACHE Progress

Offload Cross Join to Join By Server Cross join is by default join by client QUERY-TUNING (JOIN-BY-SQLDB) overrides to Join by server Performance observations Parent records are more than child records. Less record size of join tables. Increase in Join levels. Parent records are lesser than child records. Progress

Mini enhancements in OpenEdge 11.7 Dynamic query join by server optimization with ABL functions Major performance improvement in both self-service and client server Dynamic single shot outer join queries are join by client FIRST/LAST (single-shot) query enhancements Syntactical issues Performance improvements Progress

Performance Improvements : Server-side vs Client-side Time in millisec Join areas: 2to10 level joins,inner,outer and single-shot Performance Improvement with default server side execution Performance Improvement with increasing of join levels Progress

When to use Join by client Performance: Case1 Who should do the join ? 2 customers 2M orders/customer FOR EACH customer, EACH order OF customer QUERY-TUNING (NO-JOIN-BY-SQLDB): DISPLAY customer.name customer.cust-num customer.postal-code order.order-date. END. Join by client remove redundancy

When to use Join by server/Join by client Performance: Case2 Where is the join done ? Join by server is complex FOR EACH customer, FIRST order OUTER-JOIN OF customer WHERE order.order-num < 50: DISPLAY customer.name customer.cust-num customer.postal-code order.order-date. END. Join by client is slow

Server-side JOIN Limitations Join tables from multiple logical database schemas Tables to be joined that can’t find a join key Joins with a “USING” phrase EXCLUSIVE-LOCK Joins specified at the query level JOIN levels greater than 10 Progress

JOINs Best Practices Progress

Same datatype join fields Integer datatype field for join JOINs Best Practices Queries Same datatype join fields Integer datatype field for join Primary key on Join table field Indexes Make sure have index key columns to Join tables Add indexes on columns in tables regularly searched or in where clause General Use normalized tables in joins Move large objects into other non join tables Don’t use CROSS JOIN unless required Progress

JOINs Query Performance Tips

JOINs Query Performance Tips Query structure is single most likely code piece to affect performance Use index hints to ORACLE DataServer FIELD LISTs reduce network traffic –Works with NO-LOCK Use Query-tuning options when they can benefit individual query performance e.g. CACHE-SIZE,JOIN-BY-SQLDB, KeyCache_Join, No_KeyCache_Join, and LOOKAHEAD CURSOR

Future Enhancements Progress

Disclaimer This roadmap may not be interpreted as any commitment on behalf of Progress and future development, timing and release of any features or functionality described in this roadmap remains at our sole discretion. Progress

Single shot in MSS DataServer Future Enhancements Single shot in MSS DataServer Server side Index- reposition for JOINs in Oracle DataServer Server Join levels to greater than than 10 Result Set Binding for Join records Multi-schema server side Joins Server-side Joins for Adjacent-level single shots Progress

Progress