1. eSciDoc Persistence Layer Architecture

2. Page Overview 1.General eSciDoc architecture 2.Business Layer Excursion 3.Persistence Layer 4.Transformation to the module/dependency structure 2

3. Page Overview 3

4. Page 1. General eSciDoc architecture Web Application Spring Framework Persistence Layer Business Logic Layer Communication Layer Transport Layer CXF Security Interceptor Service Spec. REST Endpoint Spec. JMS Transport JAXB REST Endpoint Impl. SOAP Endpoint Spec. SOAP Endpoint Impl. HTTP Transport FastInfoSet Interceptor Service Impl. XOP Interceptor GZIP Interceptor Repository Spec. FEDORA Repository Impl. Transaction Interceptor JSON Hibernate FEDORA RMDB implements depends Transfer Objects Domain Objects Persistent Objects map 4 RMDB Repository Impl.

5. Page 1.1 Focus Web Application Spring Framework Persistence Layer Business Logic Layer Communication Layer Service Spec. Endpoint Spec. Endpoint Impl. Service Impl. Repository Spec. Repository Impl. Data Maintenance System Data Maintenance System implements depends Transfer Objects Domain Objects Persistent Objects CL-MappingPL-Mapping 5

6. Page 1.2 Mapping CL- & PL-Mappings are part of each layer  Only the layer knows, which peaces of the business data belongs to the objects of the layer Example: Hibernate  The Hibernate Layer defines the data structure to be used to persist the data (tables, indexes, etc.)  Hibernate Model A: Use one table to store Items and Container  Hibernate Model B: Use two different tables to store Items in one table and Containers in the other table Example: Fedora  Fedora uses FOXML to persist the data Example: Communication Layer (CXF)  Defines Transport-Objects which are marshalled to XML, JSON etc.  Because of the different data structures each concrete layer instance implements, it is the layers responsibility, to map the business domain objects to its own data model  The business logic is completely independent of anything that happens outside of its own borders 6

7. Page 2. Business Layer A little excursion for later reference Business Logic Layer Service Spec. Service Impl. updateItem() updateMdRecords() updateMdRecord() updateContainer() Domain Objects Item Container MdRecords MdRecord … … Persist ence Spec. Uses Domain Object Possible Interface/Call Internal call to sub-method 7

8. Page 2. Implementation structure Structured, centralized and reusable code  Sub-resource methods reused by the service-methods handling the main resources  Avoid conceptual flaws on implementation of new features, bug fixes and so on (protect the developer by design) Service methods implement the eSciDoc logic like  Update a resource: What happens, if the resource from the client equals the currently persisted resource and what happens if not? Service methods use the Domain Objects as data holders, which only implement logic, that belongs to their semantic like  getObjid() : extracts the objid from the xlink:href  getVersionNumber() : extracts the version number from getObjid() Thanks to this concept, sub-resource methods can be reused and offered as services to the communication layer as well 8

9. Page 2. Flow example 9 updateItem(arg0) updateMdRecords(arg0, arg1) updateMdRecord(arg0, arg1) updateMdRecords(mdRecords, null) updateItem(item, null) updateMdRecord(mdRecord, null) arg1 == null ? arg1 = retrieveMdRecords() arg1 = retrieveItem() arg0.hasMdRecords() & arg1.hasMdRecords() updateMdRecord(arg0.getMdRecord(name), arg1.getMdRecord(name)) arg1 == null ? arg1 = retrieveMdRecord() arg0 != arg1 ? persistMdRecord() CLPL updateMdRecords(arg0.getMdRecords(), arg1.getMdRecords()) arg0.hasMdRecord(name) & arg1.hasMdRecord(name)

10. Page 2. Loss of information  Providing central methods for reusability will cause a loss of information  eSciDoc interfaces are structured to support different resource types per interface  The centrally provided methods however do not differ between these resources, when handling sub-resources  The persistence layer may still need the information about the type of the resource If the Hibernate Model B is being used Or if the data is stored in different ways into the FOXML, depending on the resource type  Therefore we have to support the ResourceType for these methods, not matter if we need the type or not  Support future expandability and plug-in ability for the persistence layer  Signature example: updateMdRecord(MdRecord, MdRecord, ResourceType)  Support an extra package/module for the static sub-resource handlers for better understanding of the structure 10

11. Page 3. Persistence Layer Goals  Current goal for SCAPE: Use Hadoop as a persistence layer  Plug-in ability In theory eSciDoc could get released without an already implemented persistence layer How the customer can plug-in an own implementation of the persistence layer is not part of this presentation (How to do this in Tomcat, JBoss etc.)  Black box for the business layer Exception handling Mapping of the business domain objects to its own objects 11

12. Page 3. Persistence Layer as a Black Box  Exception handling Exceptions thrown in the persistence layer cannot be handled in detail by the business layer  The business layer does not know, what kind of persistence layer is being used  Plug-in support offers customers to implement their own persistence layer, we do not even know anything about their implementations The business layer may define an abstract PersistenceException, which will be the representative of all exceptions thrown in the persistence layer If we have some code in the current business layer, handling such persistence layer exceptions, we have to analyze this code and find a better solution for the idea of this code  ResourceNotFoundException Return null instead of throwing an exception in the persistence layer and throw a ResourceNotFoundException in the business layer because it is the business logic decision and design to throw an exception, if a resource was not found 12

13. Page Persistence Layer Business Layer Communication Layer 3. Exception handling – Flow example (ResourceNotFoundException) 13 retrieveItem() Retrieve Item Load data Throw 404 exception return exception XML Return Item return resource XML Resource found Delegate Persistence Exception Error-like exception IO) 404 exception OR empty result OR anything alike Return null throw PersistenceException Return resource

14. Page 3. Exception handling - Outcome  Avoid conceptual flaws on implementation Non persistence layer exceptions should not be available for the persistence layer Adjust eSciDoc module hierarchy Adjust the persistence layer interfaces to describe the expected behavior of the business layer  Implement the mapping between business domain objects and persistence layer objects as already described 14

15. Page 3. Plug-in ability Now we can add any kind of persistence layer implementation to the business layer by making them configurable in Spring 15 Business Logic Layer Service Spec. Service Impl. implements depends Domain Objects Hibernate Model A Repository Spec. Repository Impl. Data Maintenance System Data Maintenance System Persistent Objects PL-Mapping Hibernate Model B Repository Spec. Repository Impl. Data Maintenance System Data Maintenance System Persistent Objects PL-Mapping Fedora Repository Spec. Repository Impl. Data Maintenance System Data Maintenance System Persistent Objects PL-Mapping … …

16. Page 3. Plug-in ability But wait… There is something very wrong with this architecture! 16

17. Page 3. Plug-in ability Each Persistence layer defines its own specification. 17 Business Logic Layer Service Spec. Service Impl. implements depends Domain Objects Hibernate Model A Repository Spec. Repository Impl. Data Maintenance System Data Maintenance System Persistent Objects PL-Mapping Hibernate Model B Repository Spec. Repository Impl. Data Maintenance System Data Maintenance System Persistent Objects PL-Mapping Fedora Repository Spec. Repository Impl. Data Maintenance System Data Maintenance System Persistent Objects PL-Mapping … …

18. Page 3. Persistence Specification  If the specification is part of the persistence layer, the business logic has to implement a switch in order to select and work with the current persistence layer  The business logic has to implement special code, which deals with each persistence layer specification  The business logic layer depends on the persistence layer  The business logic layer cannot specifiy the expected behavior of the persistence layer (for example: the exception handling)  The persistence layer specification expects the persistent objects in its signatures and because of that the business logic has to perform the mapping from its domain objects to the persistent objects, no matter what kind of persistence layer we are using  The persistence layer is no real black box  A lot of ugly code will be written  All this conflicts with the idea to support the plug-in ability and to achieve an easy extension to use Hadoop for example 18

19. Page 3. Corrected architecture Web Application Spring Framework Persistence Layer Business Logic Layer Communication Layer Service Spec. Endpoint Spec. Endpoint Impl. Service Impl. Repository Impl. Data Maintenance System Data Maintenance System implements depends Transfer Objects Domain Objects Persistent Objects CL-MappingPL-Mapping 19 Repository Spec.

20. Page 3. Outcome  The business logic layer defines the persistence specification Uses its domain objects in the signatures Predefines the expected behavior (Exception handling etc.) Uses the PersistenceException in the signatures  Each persistence implementation implements this one specification and performs its own mapping from domain objects to its own persistent objects  The persistence layer becomes a real black box  The business logic layer can easily work with any persistence layer implementation without the need to implement special code for any persistence layer implementation  The persistence layer is pluggable like the communication layer 20

21. Page 4. eSciDoc Modules > Communication Layer 21 context-rest-impl context-rest-spec dataobjects-impl dataobjects-spec jax-rs framework om policies aspects common aa utilities spring aspects FastInfoSet geronimo-servlet 3.0 spec woodstox-core-asl CXF frontend-jaxrs Transports (HTTP, Jetty, JMS) Business logic spec. Communication Layer Business Logic Layer BL dataobjects (domain objects) Mapping

22. Page 4. eSciDoc Modules > Communication Layer 22 context-rest-impl context-rest-spec dataobjects-impl dataobjects-spec jax-rs framework om policies aspects common aa utilities spring aspects FastInfoSet geronimo-servlet 3.0 spec woodstox-core-asl CXF frontend-jaxrs Transports (HTTP, Jetty, JMS) Before we continue, we have to clean this up. Mapping

23. Page 4. eSciDoc Modules > Communication Layer 23 context-rest-impl context-rest-spec dataobjects-impl dataobjects-spec jax-rs framework om policies aspects common aa utilities spring aspects FastInfoSet geronimo-servlet 3.0 spec woodstox-core-asl CXF frontend-jaxrs Transports (HTTP, Jetty, JMS) ?  Dependencies are resolved transitively.  „aspects“ & „policies“ should be included in each root module only  Business logic modules cannot be used in the communication layer and vise versa.  Only the business logic specification should be used in module „context-rest-impl“ Mapping

24. Page 4. eSciDoc Modules > Communication Layer (Discussion)  dataobjects-impl depends on aa Reason unknown  Dependencies utilities and common Reusability  Not allowed between layers  Create a utility module outside of the layers  Dependency to om Reason unknown 24

25. Page 4. eSciDoc Modules > Communication Layer (Cleanup) 25 context-rest-impl context-rest-spec dataobjects-impl dataobjects-spec jax-rs framework om policies aspects common aa utilities spring aspects FastInfoSet geronimo-servlet 3.0 spec woodstox-core-asl CXF frontend-jaxrs Transports (HTTP, Jetty, JMS) ? Business logic spec. Communication Layer Business Logic Layer Mapping BL dataobjects (domain objects)

26. Page 4. eSciDoc Modules > Communication Layer (Final) 26 context-rest-impl context-rest-spec dataobjects-impl dataobjects-spec jax-rs framework policies aspectsspring aspects FastInfoSet geronimo-servlet 3.0 spec woodstox-core-asl CXF frontend-jaxrs Transports (HTTP, Jetty, JMS) Business logic spec. Communication Layer Business Logic Layer Mapping BL dataobjects (domain objects)

27. Page dataobjects 4. eSciDoc Modules > Communication Layer (Final) 27 context-rest-impl context-rest-spec dataobjects-impl dataobjects-spec jax-rs framework policies aspectsspring aspects FastInfoSet geronimo-servlet 3.0 spec woodstox-core-asl CXF frontend-jaxrs Transports (HTTP, Jetty, JMS) Business logic spec. See architecture on page 19 Use woodstox as the parser impl. for JAXB General servlet spec. (Also works for JBoss) ? No need to differ between spec. & impl. for objects Communication Layer Business Logic Layercall implement Mapping BL dataobjects (domain objects) international standard that specifies a binary encoding format for the XML Information Set (for JMS?)

28. Page 4. eSciDoc Modules > Business logic layer 28 Business logic spec. Business logic Impl. common policies aspects Persistence layer spec. aa Sub-resource handler See page 19 See page 7 - 14 Optional (Should be replaced with an HTTP connection or using CXF in terms of clustering abilities later) Other dependencies (no eSciDoc modules ) BL dataobjects (domain objects) implementcall

29. Page 4. eSciDoc Modules > Business logic layer (Discussion)  dataobjects-spec is required for SRU classes for example but should not be part of this layer (RequestType, ResponseType, etc.) Create dataobjects (domain objects) module for business layer, where the domain objects are being implemented and external objects are generated from XSD schema files  Code about mapping between String & DateTime should not occur in this layer (DateTimeJaxbConverter from the Communication Layer) Put such utility classes in an utility module, which can be reused on any layer and is not part of any layer Rename DateTimeJaxbConverter to DateTimeConverter (for example) 29

30. Page 4. eSciDoc Modules > Persistence layer 30 Persistence layer Impl. policies aspects Persistence layer spec. Other dependencies (no eSciDoc modules ) PL dataobjects BL dataobjects (domain objects) Mapping implement Business Logic Layer Persistent Layer

31. Page 4. eSciDoc Modules > Persistence layer (Discussion)  Nothing 31

32. Page 4. Project Structure & Dependencies 32 eSciDoc BL Project eSciDoc CL CXF ProjecteSciDoc PL Fedora Project Domain Objects BL Spec PL Spec BL Impl. Utilities

33. Page 4. Project Structure & Dependencies  Advantages The borders of the layers are clear Protects the developer to spuriously add wrong dependencies  Maybe add some kind of Maven plug-in, which evaluates the allowed eSciDoc module dependencies If a new persistence or communication layer is being implemented, a new project for this new layer can be created  Avoids adding a bunch of new modules to the current project for the new layer implementation(s)  The Utilities module has to be completely independent from any layer If that is not possible, because you may need objects from one layer, it has to be part of this layer and cannot be reused in another layer! 33

34. Page Addendum  Why do the CL, BL and PL layers looks so differently?  Let us look at this using another perspective and some examples 34 BL BL SpecPL Spec CL CXF REST CL Axis SOAP GUIPL FedoraPL Hibernate PL Hadoop OutgoingIncoming callimplement “Plug-in” Request

35. Thank you !

36. These documents are intended for presentation purposes only. Copyright lies with FIZ Karlsruhe. Any distribution or use of these documents or part thereof is subject to FIZ Karlsruhe's express approval. © FIZ Karlsruhe 2011