Architecture Grille Orientée Services ProActive Integration with HP Software in the AGOS project Towards the integration of business services within a grid environment supported by a virtualized infrastructure 2008 Sophia Antipolis – October 21 st, 2008

Architecture Grille Orientée Services 22 June 2015  Project context  Partner use cases  Draft architecture  Products mapping  HP integration  ProActive features Agenda

Architecture Grille Orientée Services Project context Putting together experts from grids, services, resources and data to define a simple, coherent and realistic solution

Architecture Grille Orientée Services Project context  Selected by the “Pôle de compétitivité Solutions Communicantes Sécurisées”  List of partners: HP, Oracle, Inria, Activeeon  Defined scope: integration project -Definition of an integration methodology -Definition of new software components -Integration of business and service dashboards -Integration of services with virtualized resources -Contribution to standards  Objective: “Integrate GRID technology into a business oriented SOA stack running over a virtualized infrastructure”

Architecture Grille Orientée Services Proposed solution  A reference architecture putting together: -business intelligence -service oriented components -grid environment -virtualized systems  A library of process and service templates based on standards  A set of tools to easily build such applications  A real-time monitoring of business processes, services and resources  A methodology to migrate to the architecture

Architecture Grille Orientée Services Applied onto water operator use case  Simulation application to better handle water distribution network and provision adequate hydraulic meshing  Focus is on software which allows the study, the planning and the control of irrigation -hydraulic simulation software for meshed networks for distribution of raw water or drinking water under pressure -full network is too large to be simulated at once; split into sub- network parts that can be studied independently  Main interest: -Parallelization of the application (SPMD approach) -Overlap multiple runs (parameters sweeping) on subnetworks -Better integrate with other applications -Better use of resources to speed up computation  Measured in terms of speed increasing of digital processing provided by the grid architecture

Architecture Grille Orientée Services Applied onto global distribution system’s usage use case  Parsing and analysis of web session logs from target applications  Focus is on providing massive data analysis: -Customer session graph analysis or other data mining services -Know how groups of customers behave: the usage / navigation in websites  Main interest -Handle the size and the volume of data generated by core applications -Parsing of logs done massively in parallel -Results need to be stored in a distributed database -Better understand how the products are used -Help to improve the products continuously through the feedback loop  Expectations -Enable an advanced workflow beyond current standards to support processes and sub processes and/or to compose hybrid processes with services. -Enable streaming results from any parallel activity on a grid to feed a distributed database -Support operational aspects like monitoring, clustering, scalability, security, load balancing and failover thus enabling high availability SLA compliance agreements to be respected. -Enables one button push start, stop and resume on any running grid process

Architecture Grille Orientée Services From building blocks to functional architecture proposal Draft architecture

Architecture Grille Orientée Services 92 June 2015 Base line…  Staged pyramid -Adaptive infrastructure of heterogeneous resources -Service delivery with quality added-value -Business intelligence based on standardized processes  Based on standards -As defined in the related standards documents -Existing implementations (proprietary / open-source)  With grid middleware -Grid environment: parallelism, distribution -Web interfaces: thin web clients access -Modeling facilities: composition, adaptive, configurable -Efficient execution: design & execution patterns

Architecture Grille Orientée Services GridGrid BI SA AI …of a Service Grid Business Intelligence Service Architecture Adaptive Infrastructure BPEL ETL SCA / GCM SOA / WS VM / XEN P2P / GTK Standards Functional

Architecture Grille Orientée Services Main building blocks  Systems Virtualization  Grid Utility Interface  Resource Manager  Tasks & Services Scheduler  Enterprise Service Bus  Distributed Shared Memory  Database Interface  Application Server Interface  Web Service Interface  SOA Registry / Repository  SOA Governance  Component Architecture  Load Balancer  Fault Tolerance  Reconfiguration Process  Business Process Execution  Real-time Monitoring Tools  Service Level Management  Business Intelligence  Reporting Tools

Architecture Grille Orientée Services Define the business processes  Processes defined in terms of business items  Cope with business processes workflow  Focus on business interactions between human and applications -How are used the applications -How are they chained in any given scenario -What is expected as an output  Not a programmatic task flow -Do not cope with service implementation details -Far from resource allocation concerns  Handle specific concepts: -Long transactions (suspended time, recurring patterns…) -Requests approval (decision making, validation…) -Human interactions (subjective perception, biased usage…) -Business ecosystem (management, billing, visualization…)

Architecture Grille Orientée Services Business Intelligence Business Process Service Level Management Reporting Business Intelligence Service Architecture Adaptive Infrastructure - Delivered results analysis - Scenario validation - Service & Service Level monitoring - Reporting on key aspects by category - Validation of performance gain, result precision, time to deliver… - Comparison with reference benchmarks

Architecture Grille Orientée Services Calling value-added services  Services stored in a registry with governance applied  Manipulating proxies to available web services  Services as composite components  With predefined characteristics and features -Interface contract -Policy enforcement -Delivered quality of service  Mapped onto: -Existing web services (black-box) -Composite web services (functional composition) -Skeleton web services (parallel pattern) -Subordinate processes (time composition) -Simple sequential services

Architecture Grille Orientée Services Implemented as grid services  Components executed on a set of resources  Skeletons providing parallel patterns for efficient execution on the grid  Transformed as jobs delegated to the grid scheduler  Optimized active objects exposed as web services on the grid  Jobs scheduled on the grid with direct access to the grid scheduler  Specialized grid schedulers dedicated for specific usage

Architecture Grille Orientée Services Service Architecture Service Composition Service Component Architecture Parallel Programming Distributed shared memory SOA Environment Repository, Registry, Orchestration Business Intelligence Service Architecture Adaptive Infrastructure - Distributed parallel services accessing data in distributed shared memory - Functional composition of service components with dynamic reconfiguration - Exposed as web services in registry with a policy enforcement contract - Executed through proxies & orchestrated by a workflow engine

Architecture Grille Orientée Services Executing on heterogeneous resources  Code execution on well-known resources or over a set of dynamically discovered resources -As tasks through a scheduler managing load-balancing, fault tolerance, high availability… -As active objects with associated features such as migration, file transfer… -As uncontrolled legacy application -As tasks delegated to another grid utility environment  Evolving set of resources (join / leave support)  With specific or adaptive characteristics (virtualized or clustered)  On a set of various operating systems  Protected or not, intrusive or not

Architecture Grille Orientée Services Resource Virtualization Grid Utility interface Resource Manager Task Scheduling Business Intelligence Service Architecture Adaptive Infrastructure - Virtual resources managed dynamically - Tasks executed on the set of evolving resources - Requests of tasks execution received via the bus

Architecture Grille Orientée Services Proposed architecture

Architecture Grille Orientée Services Products integration mapping  Global contribution -ActiveEon products from the ProActive Suite -Oracle products from the Business Services Suite -HP products from the Virtualization, Quality and SOA Centers  Alternatives from the open-source among others -ActiveBPEL, Cockpit -Sequoia -Tuscany -Xen VM

Architecture Grille Orientée Services Proposal OS, HW OS Virtualization Grid Utility interface Enterprise Service Bus Resource Manager Task Scheduling Service Composition Service Component Architecture Parallel Programming Distributed shared memory Business Process Quality Testing SLA Mgt Reporting Real-Time Monitoring Business Intelligence Service Architecture Adaptive Infrastructure SOA Environment Repository, Registry, Orchestration BPEL Engine BPEL Process Manager Integrity VM System Insight BAC Dashboard BAC Quality Center Enterprise Service Bus BIEEBPA SuiteJDeveloperDSM CoherenceJ2EE ASDB + RAC Resource ManagerTasks SchedulerGrid UtilityIC2D GCM components SOA Manager SOA Systinet Skeletons ProActive Hewlett Packard Oracle

Architecture Grille Orientée Services Open Source OS, HW OS Virtualization Grid Utility interface Enterprise Service Bus Resource Manager Task Scheduling Service Composition Service Component Architecture Parallel Programming Distributed shared memory Quality Testing Real-Time Monitoring Business Intelligence Service Architecture Adaptive Infrastructure SOA Environment Repository, Registry, Orchestration BPEL Engine Active BPEL Xen VM IC2D Apache ServiceMix EclipseSequoiaJboss / JonasMySQL Resource ManagerTasks SchedulerGtk SkeletonsTimit Apache jUDDI Cockpit RUDDITuscany

Architecture Grille Orientée Services HP Integration Points Products & Methodology

Architecture Grille Orientée Services HP contribution  Adaptive Infrastructure lab for the project  Architecture blueprint draft  Consulting & Integration methodology  HP Software products from: -SOA Center (Service Oriented Architecture) -BSM Center (Business Service Management) -VSE Center (Virtual Server Environment) -Quality Center (Quality Test Framework)  Objective: Virtualization of services and resources of the parallel / distributed grid in a business context

Architecture Grille Orientée Services AGOS blueprint draft

Architecture Grille Orientée Services Products from HP SOA Center  HP Systinet -UDDI Registry -As the de-facto SOA reference repository  HP Policy Enforcement -Policy Enforcement -As the smart SOA governance runtime  HP BAC for SOA -Services Monitoring -As the integrated SOA services monitoring -Combining business, service and resource monitoring  HP Quality for SOA -Services Testing -As the best-in-class testing environment

Architecture Grille Orientée Services Integrated specifically within AGOS  Provide a library of business processes templates including both functional aspects (gather input data, run simulation, perform pre-processing, analyze output data…) and business aspects (warn user, check execution time, request for more resources…)  Define the library of predefined SOA services  Declare all the SOA services to be used by the business processes in the SOA registry  Define and declare all SOA policy enforcements (as proxy of services) on top of existing SOA services

Architecture Grille Orientée Services Products from HP BSM Center  HP Business Availability Center -Integrated business dashboard -Reporting for business metrics at every layer Business processes Services execution Resource consumption  HP System Insight Manager -Collect raw metrics from adaptive infrastructure -Feeder of HP BAC systems for real or virtualized hardware problems

Architecture Grille Orientée Services Integrated specifically within AGOS  Define or discover the business service models to get monitored  Get in SOA registry all the services to get metrics on the architecture components  Push the indicator values into the BAC monitoring system (source, metrics name + value)  Define the SLA based on customer’s expected results  Define the reports to deliver to the business managers

Architecture Grille Orientée Services Products from HP VSE Center  HP VM -Virtualization of resources and operating systems -Adaptive deployment of resources with hypervisors -Support enterprise hardware and configurations Storage Area Network Blade Servers (Itanium, Pa-Risc, x86) Hardware & Software High Availability  HP RDP -Resource deployment protocol -Manage VM and installed software transparently  Integration expertise with Xen environment  Integration expertise with VMWare environment  Thursday 23 rd 10:30: HP VSE by Francois Donze

Architecture Grille Orientée Services Integrated specifically within AGOS  Deploy hardware (cpu, ram, disk) on demand  Deploy operating system / virtual machine or dedicated application through web services  Integrate with ProActive resource manager -Add, get or list available nodes -Define constraints of usage or quality of service Deploy within the hour Schedule the deployment of x VM Get node with x ram + y cpu

Architecture Grille Orientée Services ProActive used features Framework in action

Architecture Grille Orientée Services ProActive Suite

Architecture Grille Orientée Services Integration with ProActive ProActive Adapter X OS, HW OS Virtualization Grid Utility interface Enterprise Service Bus Resource Manager Task Scheduling Service Composition Service Component Architecture Parallel Programming Distributed shared memory Quality Testing Real-Time Monitoring Business Intelligence Service Architecture Adaptive Infrastructure SOA Environment Repository, Registry, Orchestration BPEL Engine BPEL Process Manager Integrity VM System Insight BAC Dashboard BAC Quality Center Enterprise Service Bus BIEEBPA SuiteJDeveloperCoherenceJ2EE ASDB + RAC Resource ManagerTasks SchedulerGrid UtilityIC2D Skeletons SOA Manager SOA Systinet

Architecture Grille Orientée Services Integration points with ProActive 1)Generate SCA component as a GCM component 2)Expose grid components in SOA Repository 3)Provide distributed shared memory’s data to tasks & components 4)Publish metrics from components, skeletons, tasks and resources 5)Make tasks scheduler accessible as a service 6)Expose virtual machines as resource nodes 7)Expose external grid services (like scheduler as a node resource hosting a tasks scheduler) 8)Provide IC2D aggregated metrics towards monitoring system 9)Provide introspection agent to test a particular service, component, task or object

Architecture Grille Orientée Services Thank you Questions / Answers