The ProDataSet in OpenEdge™ 10 John Sadd Progress Fellow and OpenEdge Evangelist

Agenda ProDataSet Overview Syntax Specifics and Demo ProDataSet Directions

The ProDataSet ProDataSet Data-Source1 Database Data-Source2 Field Map CustNum Name Contact Data-Source2 Field Map OrderNum CustNum OrderDate CustomerTT 1 Lift Line Skiing 2 Urpon Frisbee 3 Hoops Croquet OrderTT 6 1 01/05/93 36 1 01/19/93 79 1 02/10/93 Database Customer Lift Line Skiing Urpon Frisbee Hoops Croquet Order 1 53 01/01/93 2 81 01/04/93 3 66 01/04/93 Query… Q1 for Customer Query… Q2 for Order Data-Relation1 Event Logic Dataset:Before-fill Buffer:Before-fill Before-row-fill Row-Add Row-Delete …

The Bigger Picture – Today and Tomorrow ProDataSets become a driving force in OpenEdge Applications OpenEdge Business Logic OpenEdge .NET Interface .NET User Interface Jonas Grumby 110 Desert Isle Path Minnow, HI OK Cancel Purchase Order Business Logic Purchase Order Proxy PO ProDataSet Header Data C# Detail Data Other Data WebClient / HTML User Interface We expect the ProdataSet to be the basis for defining business objects and business logic for access from a variety of client platforms, including Progress, .NET, Java, as well as open interfaces using the Web Services protocols. 4GL or WebSpeed Interface OpenEdge 4GL Interface Jonas Grumby OK 110 Desert Isle Path Cancel Minnow, HI OpenEdge Web Services Interface 4GL

ProDataSet to ADO.NET DataSet ProDataSet is a Progress in-memory data store ProDataSet maps directly to an ADO.NET DataSet: Design a .NET interface in terms of .NET DataSets Build business logic in Progress One basic goal is to provide a direct mapping between Progress and the .NET DataSet for developers requiring access to their Progress applications from .NET.

Agenda Distributed Data Management in Version 9 ProDataSet Overview Syntax Specifics and Demo Futures

A ProDataSet is made of Temp-Tables Define temp-tables in the usual way DEFINE TEMP-TABLE ttOrder /* fields from Order db table */ FIELD OrderTotal AS DECIMAL FIELD CustName LIKE Customer.Name FIELD RepName LIKE SalesRep.RepName. DEFINE TEMP-TABLE ttOrderLine… DEFINE TEMP-TABLE ttItem… The ProDataSet is composed of already familiar Progress objects, fundamentally the temp-table. So you define your temp-tables first. We will encourage you to define your temp-tables to represent the right internal view of your data, even when it is different from the physical database storage. You can define the mapping between the two and adjust that as you change data sources or improve your database design.

Define the DataSet Define the DataSet in terms of the temp-tables it combines DEFINE DATASET dsOrder FOR ttOrder, ttOline, ttItem DATA-RELATION OrderLine FOR ttOrder, ttOline RELATION-FIELDS (OrderNum, OrderNum) DATA-RELATION LineItem FOR ttOline, ttItem RELATION-FIELDS (ItemNum, ItemNum) REPOSITION. Then you define the DataSet in terms of those tables. The DataSet can be valuable even for data requiring only one table, because of the event support and other extended features of the DataSet as compared to a temp-table.

Populating with Data-Relations …DATA-RELATION OrderLine FOR ttOrder, ttOline RELATION-FIELDS (OrderNum, OrderNum)… When a DataSet is populated, Progress retrieves children of the current parent ProDataSet OrderTT 6 1 01/05/93 36 1 01/19/93 79 1 02/10/93 When you are populating a DataSet (using the FILL method), Progress automatically interleaves reads of parent and child data when the Data-Relations describe the relationship. Again, there are events to let you intercept this to read your own set of records, filter data, etc. All the children for all the parents in the top-level query are retrieved. You can disable relations or fill at different levels to avoid filling all the data at once if that is too expensiv. Data-Relation OrderLineTT 6 1 00009 6 2 00009 6 3 00011 36 1 00009 36 2 00009 36 3 00011

Navigating with Data-Relations …DATA-RELATION OrderLine FOR ttOrder, ttOline RELATION-FIELDS (OrderNum, OrderNum)… When a DataSet is navigated, Progress filters on children of the current parent ProDataSet OrderTT 6 1 01/05/93 36 1 01/19/93 79 1 02/10/93 After you have filled the DataSet, you can navigate the data using standard Progress syntax on the temp-tables. You can also use a default dynamic query that filters each child of a relationship to records related to the currently selected parent. This can let you automatically refresh a browse on the child table, for example. Data-Relation OrderLineTT 6 1 00009 6 2 00009 6 3 00011 36 1 00009 36 2 00009 36 3 00011

Defining Data-Sources You can define a Data-Source for each temp-table buffer in the DataSet Each Data-Source can define database buffers or a query or both Data-Source is separate from any DataSet ProDataSet Data-Source SrcOrder OrderTT Query… Q1 for Order Data-Sources are separate objects so that they can be replaced, and so that the db definitions are not really part of the DataSet. The DataSet must be independent of any single data-source, both for flexibility and so that it can be passed as a parameter without db-specific definitions going along with it. Database OrderLineTT Order OrderLine Data-Source SrcOline 1 53 01/01/93 2 81 01/04/93 3 66 01/04/93 1 53 01/01/93 2 81 01/04/93 3 66 01/04/93 Query… Q2 for OrderLine

Defining a Data-Source with a Query Define a query for a top-level table For example, which Order(s) to start with Or to define a join between buffers DEFINE QUERY qOrder FOR Order, Customer, SalesRep. DEFINE DATA-SOURCE srcOrder FOR QUERY qOrder Order KEYS (OrderNum), Customer KEYS (CustNum), SalesRep KEYS (SalesRep). A Data-Source can name a Progress query that is uses, or a buffer name. In this example, the DataSet is filled with data for a specific single order, which is passed into the procedure that manages the dataSet as a parameter; thus the dynamic QUERY-PREPARE. You can combine static and dynamic objects and statement/methods as with other Progress objects. In future there will be support for associating XML documents directly with the entire DataSet. QUERY qOrder:QUERY-PREPARE ("FOR EACH Order WHERE Order.OrderNum = " + STRING(iOrderNum) + ", FIRST Customer OF Order, FIRST SalesRep OF Order").

Defining a Data-Source without a Query If there’s a Data-Relation, its fields can tell Progress how to construct a query …DATA-RELATION OrderLine FOR ttOrder, ttOline RELATION-FIELDS (OrderNum, OrderNum)… DEFINE DATA-SOURCE srcOline FOR OrderLine KEYS (OrderNum). ProDataSet OrderLineTT Data-Relation OrderTT 6 1 01/05/93 36 1 01/19/93 79 1 02/10/93 6 1 00009 6 2 00009 6 3 00011 If the Data-Relations define all the relationships, you need only open a query at the top level. Progress automatically retrieves all the related records for other tables.

Attaching Data-Sources Data-Sources are defined separately from the DataSet You can attach a particular Data-Source without changing the DataSet definition Data-Sources do not travel with the DataSet You attach a Data-Source to FILL from it or update to it, and then detach it when done. Database Data-Source SrcOrder ProDataSet OrderTT 6 1 01/05/93 36 1 01/19/93 79 1 02/10/93 Order 1 53 01/01/93 2 81 01/04/93 3 66 01/04/93 Query… Q1 for Order

ATTACH-DATA-SOURCE Method Relates a Data-Source to a DataSet buffer Optional arguments: Field mapping for fields with different names Except-list to skip unwanted fields, or Include-list to include only certain fields BUFFER ttOrder:ATTACH-DATA-SOURCE (DATA-SOURCE srcOrder:HANDLE, "Customer.Name,CustName"). BUFFER ttOline:ATTACH-DATA-SOURCE (DATA-SOURCE srcOline:HANDLE). BUFFER ttItem:ATTACH-DATA-SOURCE (DATA-SOURCE srcItem:HANDLE). The ATTACH method lets you define a field list and field mapping between Data-Source and temp-table buffer.

Populating a ProDataSet Use the FILL method to populate the DataSet FILL on the DataSet fills every table starting with top-level buffers FILL on a buffer fills that table and its descendents FILL-MODE NO-FILL – skip this table EMPTY – empty the table first APPEND – add more records to the table MERGE – add recs & eliminate dups REPLACE – replace existing recs You can FILL the entire DataSet or a subset of it. You can disable Data-relations to further control this. You can also specify via FILL-MODE whether and how to fill each individual temp-table.

Event Callback Procedures You can define event procedures for: Before and After FILL of the ProDataSet Before and After FILL of any temp-table buffer Before and After FILL of any record Use ProDataSet events to prepare queries, attach Data-Sources, etc. Use Buffer events to manipulate the table Use Record events to populate calculated fields You can use these events when there is no Data-Source at all Callback procedures let you define code for many events on the DataSet. You could connect to a db in a BEFORE-FILL event on the DataSet. You could ATTACH a data source in the BEFORE-FILL event on a buffer and DETACH it in the AFTER-FILL event. You could filter records or calculate extra fields in the BEFORE/AFTER-RECORD-FILL event.

Record-level Event Procedure Fills a calculated OrderTotal field hBuff:SET-CALLBACK-PROCEDURE ("AFTER-ROW-FILL","postRecordFill",THIS-PROCEDURE). PROCEDURE postRecordFill: DEFINE INPUT PARAMETER DATASET FOR dsOrder. hBuff = hDset:GET-BUFFER-HANDLE(“ttOrder”). FOR EACH OrderLine WHERE OrderLine.OrderNum = INTEGER(hBuff:BUFFER-FIELD("OrderNum"):BUFFER-VALUE): dTotal = dTotal + OrderLine.ExtendedPrice. END. hBuff:BUFFER-FIELD("OrderTotal"):BUFFER-VALUE = dTotal. END PROCEDURE. This example calculates an OrderTotal field for the ttOrder after the temp-table record is created and populated.

ProDataSet demo… See demo notes for a more detailed description. Most basic example of 4GL client using a dataset to read multi-table data and display. Same example but now a GUI version where data is displayed in fill-in and browse objects. Also, logic is applied to calculate the order total and to manipulate the item data.

Passing a ProDataSet as a Parameter You can pass a DataSet parameter as: DATASET – static reference like TABLE for temp-tables DATASET-HANDLE – dynamic reference to the DataSet and its definition, like TABLE-HANDLE for a temp-table HANDLE – passes a simple DataSet handle to a local procedure Note! Within a single session DataSet parameters can be passed BY-REFERENCE DATASET parameter is similar to temp-table parameter set, NOTE the However! Bullet. This allows Progress to do the most efficient thing by default both for local and distributed calls, unless specifically told to copy the DataSet. This is different from temp-table parameters, which copy the temp-table when passed as TABLE or TABLE-HANDLE.

Local DataSet Parameter RUN OrderDset.p ON hSession (INPUT iOrderNum, OUTPUT DATASET dsOrder BY-REFERENCE). ProDataSet OrderTT 6 1 01/05/93 36 1 01/19/93 79 1 02/10/93 So when run locally, a procedure references the already existing DataSet passed in as a parameter. /* OrderDset.p -- Test proc for Order Dataset */ {dsOrderTT.i} {dsOrder.i} DEFINE INPUT PARAMETER iOrderNum AS INTEGER. DEFINE OUTPUT PARAMETER DATASET FOR dsOrder.

Remote DataSet Parameter AppServer RUN OrderDset.p ON hSession(INPUT iOrderNum, OUTPUT DATASET dsOrder BY-REFERENCE). /* OrderDset.p */ DEFINE OUTPUT PARAMETER DATASET FOR dsOrder. ProDataSet OrderTT 6 1 01/05/93 36 1 01/19/93 79 1 02/10/93 ProDataSet OrderTT 6 1 01/05/93 36 1 01/19/93 79 1 02/10/93 The same procedure call done remotely copies the definitions of all tables and relations, along with all the data, from server to client or vice-versa. Client

Dynamic ProDataSets Every element of a ProDataSet can be dynamic CREATE DATASET ADD-BUFFER() ADD-RELATION() A Data-Source can also be dynamic: CREATE DATA-SOURCE And a DataSet can use dynamic temp-tables CREATE TEMP-TABLE Attributes and methods manipulate the DataSet through its handle There is a complete set of dynamic statements and methods for DataSets as for other objects such as queries and temp-tables. Attributes let you access the DataSet through its handle, and most operations on the DataSet (such as FILL) are implemented as methods on the handle.

Updating Through the ProDataSet ProDataSet tables have a companion before-image temp-table to record updates (add/change/delete) Used to pass changes to the server to be made in the Data-Source Maps to .NET support for update versions FCS of 10.0A has support for keeping track of updates to the DataSet after it has been filled.

Before-Image Table for a DataSet Original values for every modified record Initial values for every added record Original values for every deleted record ProDataSet OrderTT 6 1 01/05/93 36 1 01/19/93 79 1 02/10/93 This done using “shadow” temp-tables for each updated table that are created and maintained automatically and passed from client to server as part of the DataSet. Data-Relation OrderLineTT 6 1 00009 6 2 00010 6 3 00011 6 4 00077 0 0 00000 A 6 2 00009 M 6 5 00022 D

Saving Changes To The Database Use ATTACH-DATA-SOURCE to reattach the DataSet to the database tables for it SAVE-ROW-CHANGES method for saving a row to the database Uses the before-table for optimistic locking Allows full control in the event of a conflict ERROR, ERROR-STRING, REJECTED, DATA-SOURCE-MODIFIED attributes

ProDataSet update demo… See demo notes for a more detailed description. Most basic example of 4GL client using a dataset to read multi-table data and display. Same example but now a GUI version where data is displayed in fill-in and browse objects. Also, logic is applied to calculate the order total and to manipulate the item data.

Agenda Distributed Data Management in Version 9 ProDataSet Overview Syntax Specifics and Demo ProDataSet Directions

Enhancements in OpenEdge 10.0B and 10.0B01 COPY-DATASET method enhancements Also COPY-TEMP-TABLE Behavior changed to allow copies between non-identical tables and DataSets GET-CHANGES and MERGE-CHANGES include parents of modified rows FILL-MODE “REPLACE” Refreshes rows already in the tables Slower than MERGE because it requires FINDing each row as it is added to the temp-table

Enhancements in OpenEdge 10.0B and 10.0B01 -- continued Additional events to support data retrieval transparency OFF-END for a ProDataSet query FIND-FAILED for a FIND on a buffer Extended support for data batching BATCH-SIZE attribute for a buffer LAST-BATCH attribute signals end of data NEXT-ROWID for repositioning a query See white paper for examples: psdn.progress.com/library/product_info/ oera/index.ssp

Data batching demo… See demo notes for a more detailed description. Most basic example of 4GL client using a dataset to read multi-table data and display. Same example but now a GUI version where data is displayed in fill-in and browse objects. Also, logic is applied to calculate the order total and to manipulate the item data.

ProDataSet Features in OE10.1 Direct to-or-from XML conversion This supports ProDataSets as parameters to Web services Mapping from ProDataSets to Java SDOs REFERENCE-ONLY to use called procedure’s ProDataSet instance Also parameter passing BY-REFERENCE for temp-tables

New dynamic syntax in 10.1 Combines a dynamic ProDataSet reference with static names hDataSet:ttOrderLine.Price Simplifies logic to pass dynamic reference for flexibility

In Conclusion… The ProDataSet is a key part of the future of 4GL applications It is part of a focus on building complex data definitions and business logic in the 4GL The ProDataSet is a standards-conformant object that will integrate with .NET and Web services as part of a global Service-Oriented Architecture

Additional Resources Expert Series book: Using ProDataSets Now available through amazon.com PSDN white paper on Using Advanced ProDataSet Language Features Part of a series on implementing the OERA Instructor-led course: Using ProDataSets OpenEdge online documentation **The whitepapers are currently in .pdf format only – they are not in the DynaWeb HTML doc set (yet).

Questions?

Thank you for your time!