1. 392-"JOIN" us on DataServers Journey
June 6th Manchester, NH KLV Prasad – Principal QA Engineer Progress Software
Progress

2. Agenda DataServers Overview DataServers - JOINs
Journey of Server-side JOINs in DataServers
JOINs Best Practices
JOINs Query Performance Tips
Future Enhancements
Progress

3. Schema Holder Database
DataServers Overview
Customer1..
select * from customer
ABL Client
ABL Client
For Each Customer…
For Each Customer…
ODBC
DataServer
SQL
OpenEdge Database
OLTP
OLTP
OCI
MS SQL Server/
Oracle
Schema Holder Database
Foreign Data Source
Progress

4. DataServers - JOINs
Progress

5. DataServers OpenEdge JOINs – Evolution Join by server Join by client
Nested query
OF phrase
DataServers
Join by server
Progress

6. Join by server – How it helps in performance
Reduce the number of network turnarounds
Optimize the record matching
Reduce number of records transferred to the client
Progress

7. DataServers OpenEdge JOINs – Evolution -nojoinbysqldb
Join by client
Nested query
OF phrase
DataServers
Join by server (Default)
Static & Dynamic queries
Inner Join
Outer Join
Single-shot
-nojoinbysqldb
Progress

8. Journey of Server-side JOINs in DataServers
Progress

9. Server-side JOINs Journey – By OpenEdge(OE) Releases
Inner Join
(OE 9.x)
Outer Join
(OE 11.0)
Dynamic Query
(OE 11.2)
Single-shot
(OE 11.3)
Multiple Enhancements
(OE 11.7)
Progress

10. Left Outer Join – Pre OpenEdge 11.0
DEFINE QUERY q FOR customer, order.
OPEN QUERY q FOR EACH CUSTOMER,
EACH ORDER WHERE CUSTOMER.CUSTNUM=ORDER.CUSTNUM LEFT OUTER-JOIN.
REPEAT:
GET NEXT q.
IF NOT AVAILABLE customer THEN LEAVE.
DISP CUSTOMER.CUSTNUM ORDER.ORDERNUM.
END.
1117 records
3953 records
SELECT /*+ INDEX_ASC(T0 CUSTOMER##CUSTNUM)*/ * FROM CUSTOMER T0
SELECT /*+ INDEX_ASC(T0 ORDER_##CUSTORDER)*/ * FROM ORDER_ T0 WHERE (CUSTNUM = :1)
Client side processing
1117*3953
3953 rows
Progress

11. Left Outer Join – OpenEdge 11.0
Join by client
3953 records
1117 records
SELECT /*+ INDEX_ASC(T0 CUSTOMER##CUSTNUM) */ * FROM CUSTOMER T0
SELECT /*+ INDEX_ASC(T0 ORDER_##CUSTORDER) */ * FROM ORDER_ T0 WHERE (CUSTNUM = :1)
Client side processing
1117*3953
Supported in Oracle and MSS DataServer
Cost based optimizer of foreign Database is utilized in making query access plan
OUTER-JOIN request is served by USE_MERGE hint
Smaller result set is sent to client for a single database request
Handling NULL Joins -JOIN-ON-SELECT, SELECT-ON-JOIN
3953 rows
Join by server
3953 records
SELECT /*+ USE_MERGE (T0,T1) */ T0.*, T1.* FROM CUSTOMER T0 LEFT OUTER JOIN ORDER_ T1 ON (T1.CUSTNUM = T0.CUSTNUM)
Client result set
3953 rows
Progress

12. Server-side Join for Dynamic Queries - OpenEdge 11.2
Dynamic query
Server-side Join for Dynamic Queries - OpenEdge 11.2
hQry:QUERY-PREPARE("FOR EACH CUSTOMER , "
+ "EACH ORDER outer-join of customer").
hQry:QUERY-OPEN().
REPEAT WHILE NOT hQry:QUERY-OFF-END.
hQry:GET-NEXT().
IF NOT AVAILABLE customer THEN LEAVE.
DISPLAY Customer.CustNum Customer.NAME Order.OrderNum.
END.
Static queries are by default Join by server
Implemented in OpenEdge 11.2 for Oracle DataServer and MSS DataServer
Supports all types of joins (inner, outer and self)
3953 records
SELECT /*+ USE_MERGE (T0,T1) */ T0.*, T1.* FROM CUSTOMER T0 LEFT OUTER JOIN ORDER_ T1 ON (T1.CUSTNUM = T0.CUSTNUM)
Client result set
3953 rows
Progress

13. Server-side single-shot – Pre OpenEdge 11.3
FOR EACH customer, FIRST order
WHERE customer.custnum = order.custnum:
DISP customer.custnum order.ordernum.
END.
1117 records
3953 records
SELECT /*+ INDEX_ASC(T0 CUSTOMER##CUSTNUM) */ * FROM CUSTOMER T0
SELECT /*+ INDEX_ASC(T0 ORDER_##CUSTORDER) FIRST_ROWS */ * FROM ORDER_ T0 WHERE (CUSTNUM = :1) ORDER BY CUSTNUM, ORDERNUM
Client process
1117 rows
Progress

14. Server-side single-shot – OpenEdge 11.3
Supported only Oracle DataServer
Correlated subquery support added in OpenEdge 11.4
Query performance is good if records are evenly distributed in child table for parent table key
Join by client is efficient only if child table has tens of thousand matching records for a few parent records
Limitations
Date Field in WHERE clause
Program variable used in WHERE clause
BY clause
Multiple FIRST/LAST options in the query
1117 records
3953 records
SELECT /*+ INDEX_ASC(T0 CUSTOMER##CUSTNUM) */ * FROM CUSTOMER T0
SELECT /*+ INDEX_ASC(T0 ORDER_##CUSTORDER) FIRST_ROWS */ * FROM ORDER_ T0 WHERE (CUSTNUM = :1) ORDER BY CUSTNUM, ORDERNUM
Join by client
Client process
1117 rows
Join by server
SELECT customer*,Order* FROM Customer T0, Order T1 WHERE Customer.CustNum = Order.CustNum AND Order.OrderNum = (SELECT MIN (T1.OrderNum) FROM Order WHERE Order.CustNum = Customer.CustNum)
1117 records
Client result set
1117 rows
Progress

15. Statement level lock Enhancements- OpenEdge 11.7
MSS DataServer
Statement level lock Enhancements- OpenEdge 11.7
Join by server for MSS DataServer (OE 11.7)
Lock at record level with benefit of NO-LOCK array fetch
Query open with NO-LOCK,SHARE-LOCK – Join by server
Query open with EXCLUSIVE-LOCK – Join by client
Performance switches
QUERY-TUNING (KEYCACHE-JOIN),(NO-KEYCACHE-JOIN)
-Dsrv PRGRS_JOIN_NOKEYCACHE
Progress

16. Offload Cross Join to Join By Server
Cross join is by default join by client
QUERY-TUNING (JOIN-BY-SQLDB) overrides to Join by server
Performance observations
Parent records are more than child records.
Less record size of join tables.
Increase in Join levels.
Parent records are lesser than child records.
Progress

17. Mini enhancements in OpenEdge 11.7
Dynamic query join by server optimization with ABL functions
Major performance improvement in both self-service and client server
Dynamic single shot outer join queries are join by client
FIRST/LAST (single-shot) query enhancements
Syntactical issues
Performance improvements
Progress

18. Performance Improvements : Server-side vs Client-side
Time in millisec
Join areas: 2to10 level joins,inner,outer and single-shot
Performance Improvement with default server side execution
Performance Improvement with increasing of join levels
Progress

19. When to use Join by client
Performance: Case1
Who should do the join ?
2 customers
2M orders/customer
FOR EACH customer, EACH order OF customer
QUERY-TUNING (NO-JOIN-BY-SQLDB):
DISPLAY customer.name customer.cust-num customer.postal-code order.order-date.
END.
Join by client
remove redundancy

20. When to use Join by server/Join by client
Performance: Case2
Where is the join done ?
Join by server is complex
FOR EACH customer, FIRST order OUTER-JOIN OF customer WHERE order.order-num < 50:
DISPLAY customer.name customer.cust-num customer.postal-code order.order-date.
END.
Join by client is slow

21. Server-side JOIN Limitations
Join tables from multiple logical database schemas
Tables to be joined that can’t find a join key
Joins with a “USING” phrase
EXCLUSIVE-LOCK Joins specified at the query level
JOIN levels greater than 10
Progress

22. JOINs Best Practices
Progress

23. Same datatype join fields Integer datatype field for join
JOINs Best Practices
Queries
Same datatype join fields
Integer datatype field for join
Primary key on Join table field
Indexes
Make sure have index key columns to Join tables
Add indexes on columns in tables regularly searched or in where clause
General
Use normalized tables in joins
Move large objects into other non join tables
Don’t use CROSS JOIN unless required
Progress

24. JOINs Query Performance Tips

25. JOINs Query Performance Tips
Query structure is single most likely code piece to affect performance
Use index hints to ORACLE DataServer
FIELD LISTs reduce network traffic –Works with NO-LOCK
Use Query-tuning options when they can benefit individual query performance e.g. CACHE-SIZE,JOIN-BY-SQLDB, KeyCache_Join, No_KeyCache_Join, and LOOKAHEAD CURSOR

26. Future Enhancements
Progress

27. Disclaimer
This roadmap may not be interpreted as any commitment on behalf of Progress and future development, timing and release of any features or functionality described in this roadmap remains at our sole discretion.
Progress

28. Single shot in MSS DataServer
Future Enhancements
Single shot in MSS DataServer
Server side Index- reposition for JOINs in Oracle DataServer
Server Join levels to greater than than 10
Result Set Binding for Join records
Multi-schema server side Joins
Server-side Joins for Adjacent-level single shots
Progress

29. Progress