1 Tutorial Outline 30’ From Content Management Systems to VREs 50’ Creating a VRE 80 Using a VRE 20’ Conclusions

From Content Management Systems to VREs: requirements, challenges, and opportunities Pasquale Pagano CNR-ISTI RCDL October 2007 Dubna (Russia)

3 Session Outline Towards VREs  A bit of history  Delos DL Reference Model  VREs, VOs, and e-Infrastructures  D4Science Vision gCube System  Characteristics  Technological Complexity and Solution  Architecture gCube Standards and Technology  SOI, WSRF  gCore gCube Enabling Services  Features  Opportunities From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

4 The evolution Virtual Research Environments many virtual organizations Repository + Catalogue + Search service few large institutions 1996 consumer Digital Library few small institutions 2001 consumer 2006 consumer and resource provider Digital Library Management System many small institutions 2003 consumer and data provider From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

5 The DL Universe DIENST ADEPT NSDL NDLTD DSPACE FEDORA TEL DRIVER BRICKS DILIGENT ACM DL ECHO PERSEUS OPENDLIB EUROPEANA From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

6 DELOS: Digital Library definition A (potentially virtual) organization that comprehensively collects, manages, and preserves for the long term rich digital content and offers to its user communities specialized functionality on that content, of measurable quality, and according to prescribed policies * DELOS Reference Model for Digital Libraries From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

7 DL Reference Model: the 7 Domains The Reference Model is founded on 6+1 domains: Resource – captures generic characteristics (super-domain) Content – information available User – actors interacting with system Functionality –operations supported Policy – rules and conditions governing operation Quality – qualitative & quantitative characterisations of system Architecture –physical software (and hardware) constituents concretely realising the DL * DELOS Reference Model for Digital Libraries From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

8 DL Reference Model: the 7 Domains [cont.] * DELOS Reference Model for Digital Libraries From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

9 Reference Frameworks From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

10 Virtual Research Environments (VRE) Distributed frameworks for carrying out cooperative activities like “in silico experiments”, data analysis and processing, production of new knowledge using specialized tools Largely based on retrieval and access of always updated knowledge from diverse heterogeneous content sources Produce knowledge that is preserved and made available for other usages inside and outside the VRE From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

11 Virtual Research Environments: characteristics Highly dynamic, created and dismissed on-demand Based on specialised tools which support the generation of new knowledge M26 0 0,2 0,4 0,6 0,8 1 1,2 Informaion Service Broker & Matchmaker Keeper DVOS VDL Generator Content Management Wrapper & Monitor Content Security Metadata Broker Annotation Metadata Management Data Fusion CSDS Personalization Index Service Search Service Feature Extraction Service Process Design & Verification Process Execution & Reliability Process Optimization Arte Portal ImpECt Portal PrototypeAvailableBuild Operating on new information objects From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

12 Virtual Research Environments: the vision A Virtual Research Environment (VRE) provides a framework of applications, services and data sources dynamically identified to support the underlying processes of research/collaboration/cooperation. The purpose of a VRE is to help researchers* belonging to Virtual Organization by managing the increasingly complex range of tasks involved in carrying out their activities. * Researcher has to be considered in the large, i.e. end-user, decision-makers, resource and data providers, etc. From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

13 Virtual Organization (VO) A Virtual Organization (VO) models sets of users and resources belonging to a e-Infrastructure. It defines clearly and carefully  what is shared,  who is allowed to share,  and the conditions under which sharing occurs, usually based on an authentication and authorization policies. VOs may have a limited lifetime and they are dynamically created to satisfy transient needs of the constituent potentially heterogeneous actual Organizations. From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

14 e-Infrastructure An infrastructure is the basic physical and organizational structures and facilities (roads, power supplies,..) needed for the operation of a society or enterprise An e-Infrastructure provides support for effective consumption of shared resources:  hardware-bound resources (i.e. networks, storage, instruments, and computational resources),  system-level software resources (i.e. basic middleware services),  and application-level software resources (i.e. data sources and services). These e-Infrastructures offer mechanisms that concurrently exploit networks, grids and data in a seamless fashion, and will thus enable scientific communities to operate within a coherent model, regardless of the location of their research facilities. From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

15 D4Science vision calls for the realization of scientific e-Infrastructures that will remove technical concerns from the minds of scientists, hide all related complexities from their perception, and enable users to focus on their science and collaborate on common research challenges gCube is a framework to manage distributed e-infrastructures where it is possible to define, host, and maintain dynamic virtual environments capable to satisfy the collaboration needs of distributed Virtual Organizations (VOs) From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

16 gCube System From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

17 gCube Characteristics A Software framework  to support ON-DEMAND virtual collaborations* among remote parties  cost-effective, secure,  dynamic, both short and long lived  overcome ad-hoc systems alike  to make discoverable and accessible  computing, storage,  data and service resources  to promote and/or contribute to data and service integration * Research Environment From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

18 gCube Technological Complexity Software framework  needs a ‘middleware’ (typically distributed)‏  is open by definition  new resource types and/or new resource instances can be de/registered at any time  is powerful if it supports application scope  the portion of the infrastructure in which a resource exists  the portion of the infrastructure in which a resource can act, operate, or has power or control  is powerful if it supports sharing scope (controlled resource sharing)  machines, storage, data and services resources From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

19 gCube Technological Solution By relying on gLite, gCube is an e-Infrastructure enabling system to share  computing, storage, data and service resources → g3 gCube system allows collaborations in eScience  strongly content-oriented, potentially data and processing intensive  within the sharing scope of Virtual Organizations (VOs)‏  broader and longer lived  may stretch across the whole infrastructure  or else over significant subsets thereof  take place in Virtual Research Environments (VREs) scope  interactively created, managed, defined, and used: system administrators, application designers, researchers  typically short to medium lifespan From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

20 gCube Architecture gCube framework is composed by 4 main subsystems:  Enabling Services  definition and runtime management of VREs  Information Organization Services  storage, management, description, and annotation of information in a VRE  Information Retrieval Services  retrieval of information in a VRE  Presentation Services  VRE users interface with system and application services From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

21 gCube Standards and Technology From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

22 Is a Virtualised IT Infrastructure which 1.exposes a catalog of services instead of running service instances, 2.supports Workflow definition and execution, and 3.includes infrastructure resources such as compute, storage, and networking hardware and software (middleware) to support the running of services. Is a Virtualised IT Infrastructure which 1.exposes a catalog of WS instead of running service instances, 2.supports SOA Application, and 3.includes infrastructure resources such as compute, storage, and networking hardware and software (middleware) to support the running of services. Service Oriented Infrastructure gCube provides a production quality software framework to enable scientific e-Infrastructures empowered by a collaborative environment From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

23 gCube & Standards for communication Java WSCore, Apache Axis, and GridForum specifications (and implementation if any):  WS-Notification, WS-Addressing, WS-Security  WSRF  WS-ResourceProperties (WSRF-RP)‏  WS-ResourceLifetime (WSRF-RL)‏  WS-BaseFaults (WSRF-BF)‏  WS-ServiceGroup (WSRF-SG)‏  WS-DAI, WS-DAIR, WS-DAIX From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

24 Web Service Resource Framework (WSRF) Web Services Life time Properties Service Groups Notifi- cation Error Handling Unified way to model and interact with stateful web services Logic WS Logic Statefull WS State From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

25 Web Service Resource Framework (WSRF) Lifetime (WS-ResourceLifetime)  Factory based dynamic creation of services  Instances are created with a limited lifetime  Prevent services from consuming resource indefinitely (“Garbage Collection”) Properties (WS-ResourceProperties)  Defines type and values of a resource state Web Services Life time Properties Service Groups Notifi- cation Error Handling Service groups (WS-ServiceGroups)  Describes an interface for operating on collections of WS-Resources  E.g. to distribute an action to a set of services Notification (WS-Notification)  Notification about state changes  Applies traditional publish/subscribe paradigm Error Handling (WS-BaseFaults)  Defines base handling of communication errors From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

26 WSRF: Advantages and Disadvantages Advantages  Clear Description of Resources and Interfaces  Dynamic sharing of resources  On-demand services exploitation  Cross-organizations trusted environment  Widely accepted Web Service standards Disadvantages  Reference implementations are still in development  Several complementing specifications are in development  Complex middleware requires maintenance and administration overhead From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

27 Towards the usability: gCore To overcome the complexities of the design and implementation of SOI compliant services  an application framework for the consolidation / development of existing/new gCube services  the gCore Framework (gCF)‏ To meet the needs of system administrators, infrastructure managers, and resource providers  an easy to install self-contained service container for the participation to Service Oriented Infrastructures  the gCore Container (gCore)‏ From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

28 gCF facilities: partial overview WS management lifetime activation update re-deployement failure State configuration models lifetime scoping publication persistence notification Calls scoping faults security scoping serviceport-types From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

29 gCube & Standards for communication Mutual authentication based on GSI secure conversation (through delegation and renewal)‏ Business Process Execution Language for Web Services (WS-BPEL)‏ GridFTP and SRM support VOMS for users and groups management GWT and JSR168(JSR268 is coming)‏ infrastructures.eu/quality/index.php/Standards From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

30 gCube Enabling Services From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

31 gCube Enabling Services VRE Management services  definition of VREs  the dynamic deployment of VRE resources across the infrastructure Software Repository service  Storage and provision of deployable software components Information Service  publication of resources profile  discovery of VRE resources through xPath, XQuery  real-time monitoring  subscription/notification  Dynamic Virtual Organisation Support services  robust and flexible security framework for managing VOs From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

32 HW gHN HW Service provision continuity HW gHN HW Balancing utilization with head room Dynamic Load Balancing WS State WS State CPU Usage 30% 90% Rapid deployment Production HW gHN Software Repository WS … WS State WS State WS State WS State gCube Enabling Services Soon available Failure Recovery From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

33 gCube goes beyond other management systems by adding Service Management capabilities  Provides solutions for system administrators to  eliminate manual deployment overheads,  eliminate manual environment configuration overheads,  ensure optimal placement of services within the infrastructure  support user community services orchestration  Opens unique opportunities for outsourcing state-of-the-art service implementations gCube Enabling Services From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

34 Conclusions gCube infrastructures creates new opportunities to change the VRE development model used by distributed and dynamic organisations and communities gCube offers  an application framework for the development of Stateful Web Services (gCF)  an easy to install self-contained service container (gCore)  SOI middleware (Enabling services)  Rapid deployment  Failure recovery  Load balance (soon) From Content Management Systems to VREs 10 October 2008, Dubna (Russia)

35

36 gCube & Standardization Bodies ISO: data representation (e.g. ISO3166 for countries, ISO4217 for currencies) and metadata (ISO19115 for GIS)‏ OGF: Standards related to Architecture (e.g. OGSA), Data (e.g. DAIS, GridFTP), Management (e.g. GLUE, Resources Usage), Applications (e.g. DRMAA), Compute (e.g. JSDL)‏ OAI: Resources Exposure/Harvesting (OAI-PMH) Resources Exchange (OAI-ORE)‏ OASIS: Standards related to stateful web services (e.g. WSRF), process management (BPEL), remote user interfaces (WSRP), A&A (SAML / XACML)‏ W3C: All the standards related to Web Architecture (e.g., URI/URL, HTTP), Service Oriented Architectures (e.g. SOAP, WSDL, WS- Addressing) and data representation and manipulation( e.g. XML*) Others: Classification systems (e.g. ISSCAAP, ISSCFV, ISSCFG), features representation (e.g. GML for GIS), metadata (e.g. AgMES for Agricultural, SDMX for Statistics)‏ From Content Management Systems to VREs 10 October 2008, Dubna (Russia)