1. ChangeSummary / DAS Requirements (SDO 3.0 Virtual F2F) Christophe Boutard (christophe.boutard@datadirect.com) François Huaulmé (francois.huaulme@datadirect.com) Date: February 2009

2. Agenda SDO + DAS current states ChangeSummary scope ChangeSummary operations clarification New ChangeSummary format 2

3. Today SDO… … provides a ChangeSummary tracking modifications in the scope of a containment tree Changes done on objects that are not in the data graph closure are not taken into account Containment relationships match XML oriented Data Models … considers creations/deletions/modifications in scope of the data graph (containment tree) … has the XML ChangeSummary format implying data graph XML serialization … will introduce the “Orphans” concept in v3.0: Orphans allow dealing with non-closed data graphs (shared instances, huge business oriented models…) Orphans only exist in the XML (serialization artifacts) 3

4. The upcoming DAS… … will provide data oriented services for reading SDOs and applying back changes to multiple backends (RDBMS, XML, services…) … defines an « applyChanges » operation relying on the SDO ChangeSummary content DAS can be seen as the main consumer for ChangeSummary … will be able to deal with non-closed data graphs Complex Business/Data models support Relationships between multiple « read » services results Non-closed data graphs changes committed in a single transaction 4

5. Agenda SDO + DAS current states ChangeSummary scope ChangeSummary operations clarification New ChangeSummary format 5

6. Objective Extend the XML ChangeSummary scope Put the « Orphan » objects changes in the scope of the XML ChangeSummary Allow DASes to apply changes done on non-closed data graphs Complete SDO 3.0 non-closed graphs support 50% already completed (« orphans » data objects XML serialization) 50% TO BE DONE (« orphans » changes) Avoid breaking changes with SDO 2.1.x 6

7. How does it work? (1/2 ) 3 scenarios: Orphans outside the scope of a ChangeSummary continue to be serialized without changes as described in the first CD Orphans in the scope of a ChangeSummary having « logging=false » follow the rule above even if the main containment tree tracks changes Orphans in the scope of a ChangeSummary having « logging=true » are serialized and their changes included into the XML ChangeSummary DAS « read » services return data graphs with a logging=true ChangeSummary 7

8. How does it work? (2/2) In-memory orphans are just objects that are outside the containment tree to serialize At XML serialization time, orphan objects are added to orphan properties and then belong to the containment tree Orphans having changes are included in the XML ChangeSummary and can be referenced like any DataObject in SDO 2.1.x 8

9. XML ChangeSummary: Model example 9 Company Employee Address Project Country employees * employeeOfTheMonth project address country 1 1 1 1 containment non-containment

10. XML ChangeSummary: SDO 2.1.x scope 10 Company Employee Address Project Country employees * employeeOfTheMonth project address country 1 1 1 1 containment non-containment

11. XML ChangeSummary: SDO 3.0 scope 11 Company Employee Address Project Country employees * employeeOfTheMonth project address country 1 1 1 1 containment non-containment addressOrphans projectOrphans

12. XML ChangeSummary: proposed scope 12 Company Employee Address Project Country employees * employeeOfTheMonth project address country 1 1 1 1 containment non-containment addressOrphans projectOrphans

13. Example DataObject megaCorp; megaCorp = das.readCompanyByName(“MegaCorp”); List employees = megaCorp.getList(“employees”); DataObject john =employees.get(0); DataObject mary = employees.get(1); DataObject jane = employees.get(2); DataObject address = john.getDataObject(“address”); DataObject country; country = address.createDataObject(“country”); country.setString(“name”, “UK”); DataObject project = mary.getDataObject(“project”); project.setString(“description”, “NEW description of PRJ002”); 13

14. Example: XML Schema <xsd:element name="addressOrphans“ type="company:AddressType“ minOccurs="0“ maxOccurs="unbounded” sdox:orphanHolder="true"/> <xsd:element name="projectOrphans" type="company:ProjectType" minOccurs="0" maxOccurs="unbounded“ sdox:orphanHolder="true"/> <xsd:attribute name="employeeOfTheMonth" type="xsd:string" sdo:propertyType="company:EmployeeType"/> 14

15. Example: XML data graph <company:company name="MegaCorp" employeeOfTheMonth="#/sdo:datagraph/company:company[1]/employees[2]"> <employees name="John Jones" SN="E0001" address="#/sdo:datagraph/company:company[1]/addressOrphans[1]“ project="#/sdo:datagraph/company:company[1]/projectOrphans[1]"/> <employees name="Mary Smith" SN="E0002" address="#/sdo:datagraph/company:company[1]/addressOrphans[1]“ project="#/sdo:datagraph/company:company[1]/projectOrphans[2]"/> <employees name="Jane Doe" SN="E0003" address="#/sdo:datagraph/company:company[1]/addressOrphans[2]“ project="#/sdo:datagraph/company:company[1]/projectOrphans[2]"/> 15

16. Example: XML ChangeSummary <changeSummary delete="" create="#/sdo:datagraph/company:company[1]/addressOrphans[1]/country[1]"> <company:company sdo:ref="#/sdo:datagraph/company:company[1]" employeeOfTheMonth="#/sdo:datagraph/company:company[1]/employees[1]"> 16

17. Agenda SDO + DAS current states ChangeSummary scope ChangeSummary operations clarification New ChangeSummary format 17

18. Objectives Make events recorded in the ChangeSummary more easily usable by a DAS Objects lifecycle should be independent of the data graph « Creations » and « Deletions » should describe the state of objects with a bigger scope than a data graph Distinguish « moving » an object from « deleting » it 18

19. Extending the DataObject lifecycle SDO 3.0 and DAS tend to consider DataObjects outside data graphs The SDO 3.0 Keys lead to extend the DataObjects lifecycle DataObjects can be identified independently from a data graph appear in multiple data graphs In SDO + DAS architecture objects uniqueness must be ensured at the backend level isCreated in a data graph does not always mean isCreated at the backend level 19

20. Creations (1/2) In DAS architectures DataObjects can be instantiated by: User (client side): insert new instances into the backend DAS (server side): “read” operations results Today “ChangeSummary.isCreated()” means “is added to a data graph” This does not allow differencing objects instantiated by users (really created) from objects returned by a DAS SDO must not solve DAS issues but should provide mechanisms that make things possible… 20

21. Creations (2/2) Containment relationships would continue following the SDO 2.1.x rules Adding an object on a containment relationships would be tracked as a creation without considering previous actions done on the object (e.g. DataObject.delete()) Non containment relationships to orphan objects need a new rule An object added to a non containment relationship will not be considered as a creation if it was not in the scope of a ChangeSummary at the time ChangeSummary.beginLogging() was called 21

22. Delete vs. Move Today SDO does not allow distinguishing “moves” from “deletes” From a DAS (backend) point of view “Delete” must produce a delete order (e.g. SQL delete for RDBMS) “Move” must produce a reference modification Explicit deletions (DataObject.delete()) must always be tracked as delete orders without considering the containment information The needed clarification only affects “moves”… 22

23. How to track “Moves”? Containment relationships Moving a contained DataObject should continue to be tracked as a deletion in the context of the old container Then setting the moved object to another relationship would follow the rules defined for “creations” Non containment relationships to “orphan” objects A “nullification” must be tracked for the old owner of the relationship The moved object is not marked as deleted 23

24. Agenda SDO + DAS current states ChangeSummary scope ChangeSummary operations clarification New ChangeSummary format 24

25. Objectives – Questions Objectives: Provide a more efficient XML ChangeSummary format Provide a format that can be more easily used by a DAS SDO Keys mapped to a backend identity Optimistic Concurrency Control (OCC) based on old values + Keys Questions: Should SDO 3 allow a ChangeSummary to be sent without anything but only the changes? Avoid data graph XML serialization (less verbose). Currently XML ChangeSummary only stores old values. New values are serialized in the data graph. A more efficient XML ChangeSummary format could leverage SDO 3 Keys Objects identity can move from technical state to a business representation Use of identity in sdo:ref? 25

26. How to represent the object identity? Use SDO 3 identity in the ChangeSummary. But how? A canonical representation of the key? General format: {type_name, idProperty0=idProperty0Value, …, idPropertyN=idPropertyNValue} Examples sdo:ref=”{abc:Type, id=2, id2=3, objectId={abc:ObjectType, objId=10}}” sdo:ref=”{abc:Type, id=2, id2=3}” A new reprensentation of the “sdo:ref”? XML elements containing Keys Something else? 26

27. Improve SDO ChangeSummary information SDO 2.1: ChangeSummary is the “copy” of a data graph when beginLogging() was called. Should ChangeSummary be… An action log? A list of actions (new, move, delete, set…) Can highly reduce the old values size when dealing with many=true properties. An optimized action log? A log that prunes unnecessary recorded actions. 27

28. QUESTIONS ? 28