VL-e PoC Architecture and the VL-e Integration Team David Groep VL-e work shop, April 7 th, 2006

The VL-e PoC: Proof-of-Concept What is the PoC Environment? A shared, common environment, where different tools and services are both used and provided by the VL-e community The basis for further application development

Grid Middleware Networking Network Service (lambda networking) Application specific service Application Potential Generic service & Virtual Lab. services Grid & Network Services Virtual Laboratory VL-e Experimental Environment VL-e Proof of concept Environment App1 App 2 App 3 Virtual Lab. rapid prototyping (interactive simulation) Additional Grid Services (OGSA services) PoC Position in the VL-e structure

PoC R1 Architectural Vision Many different tools and services … … from many different providers … … tied together in a common framework: Service-Oriented Architecture (SOA) Aim for re-useable software components Integrated at the work-flow layer Needs commitment from everyone to work on and with the PoC and in this shared environment

Service Orientation All functions defined as services All services are independent  Operate as "black boxes"; o external components neither know nor care how boxes are executed o merely that they return the expected result. The interfaces are invocable At an architectural level, it is irrelevant whether o they are local or remote o what interconnect scheme or protocol is used to effect the invocation, &c In the VL-e and Grid environment, these invocations are across different organisations and domains from: Adam Belloum, Introduction to Grid Computing , UvA Master of Grid Computing

Service Orientation from: Adam Belloum, Introduction to Grid Computing , UvA Master of Grid Computing ‘non-SOA architecture’‘service oriented’ ‘Grid Services’ are services that operate on stateful resources

Service composition Services in the architecture represent basic (but not necessarily atomic) functionality Services can also be ‘general purpose’  ‘actors’ in work flow system run legacy applications  use such ‘general purpose’ services for high-through or high-performance tasks  don’t just ‘servicize’ everything you see Work flow should be the integrative layer of VL-e  means functionality should be invocable as a service  work flow (graphical) systems help in composition but are not the only way to interact with services

PoC Environment The PoC distribution contains components to enable service-oriented development enable application development provide access to computing, storage, and information systems  distributed geographically driven by the VL-e application scenarios (use cases) stable base environment, with managed releases today R1 is (still) mostly a ‘bag of components’

VLeIT – the VL-e Integration Team Steering PoC Development and the VL-e Architecture

VLeIT members Scientific Director (chair)Bob Hertzberger Applications liaisonRobert Belleman Adapt. Inform. DisclosureMachiel Jansen WorkflowZhiming Zhao, Paul van Hooft, Adam Belloum Information ManagementNader Mirzadeh, Hakan Yakali, Paul van Hooft VisualizationRobert van Liere Problem Solving EnvironmentsBreanndán Ó Nualláin HPCN and RP representativeKees Verstoep Scaling & ValidationDavid Groep PoC infrastructureMaurice Bouwhuis

VLeIT Activities Application scenario (‘use case’) coordination  Collecting usage scenarios  Correlate across SP’s and identify common elements  Advising the application scenario groups Define the content of the PoC Distribution  focus on tools that are shared between SP’s  incorporate tools that are the basis for our SOA model Management of the scientific-programmer pool  prioritization of the development effort  driven by the application scenario groups Outputs are Recommendations to the Directorate

Application Scenario Groups Scenarios proposed & defined by the application SPs Small ‘design team’ (3-4 persons)  from application ICT experts and selected P2 experts  comes up with a design/sequence diagram  identifies existing relevant components  does a gap-analysis  proposes the work plan to completion VLeIT compiles the results  assigns scientific programmers  updates the PoC environment with relevant components or new developments

Application Scenario Status ~ 20 scenarios submitted Subset prioritized for the R1 timeframe  Prioritization based on resource availability  By VLeIT in consultation with the applications Some have been completed  High-throughput & parallel processing of R  … Several ongoing Targeted at or intended for the PoC Environment …see Adam Belloum’s presentation this afternoon

Defining content of the PoC Distribution Application development Matrix clusters Central Storage (SRB, dCache/SRM) Distributed Clusters, SURFnet Initial compute platform Stable, reliable, supported releases of the Grid MW & VL-software VL-e Proof of Concept Environment VL-e Rapid Prototyping Environment DAS-2, local resources VL-e Certification Environment NL-Grid Fabric Research Cluster Test & Cert. Grid MW & VL-software Compatibility Flexible test environment Environments Typical usage ‘keywords’ Virtual Lab. rapid prototyping Flexible, ‘unstable’ Download Repository PoC Installer Common repository Integration tests stable, tested releases Tagged Release Candidates External software VLeIT Recommendation Point

Elements in the PoC The PoC refers to three distinct elements PoC Software Distribution  set of software that is the basis for the applications  both grid middleware and virtual lab generic software PoC Environment  the ensemble of systems running the Distribution  including your own desktops or local clusters/storage PoC Central Facilities  those systems with the PoC Distribution centrally managed for the entire collaboration  computing, storage and hosting resources

The PoC software distribution The PoC software suite. For convenience, the following elements of this suite can be distinguished:  Grid foundation middleware; the basic software that is based on interfaces and concepts that are globally adopted and standardized. This includes elements such as the security model, resource allocation interface, …  Generic Virtual Laboratory software; the software developed within the project for the PoC.  Services imported from outside; given that not all services are necessarily developed within VL-e, some will be imported.  Associated installation and deployment tools; the PoC suite is installed on the central facilities and (where applicable) also available for distributed installation.

PoC R1: Questions and Support Each software component has an ‘expert’ on VLeIT to work with you on its optimal use or deployment coordinate enhancement requests see under “software” for a list Availability Download: On DVD Via a trial run in a virtual machine (image on the DVD)

In Summary PoC provides common basis for applications Driven by the application scenarios Designed for your (application) purposes Managed via the VLeIT Team