Cracow Grid Workshop, November 5-6, 2001 Towards the CrossGrid Architecture Marian Bubak, Marek Garbacz, Maciej Malawski, and Katarzyna Zając

Cracow Grid Workshop, November 5-6, 2001 Outline –Introduction –CrossGrid Architecture Team –Layered architecture –Software integration –Standard operational procedures –Collaboration with DataGrid –Conclusion

Cracow Grid Workshop, November 5-6, 2001 Introduction –Layered structure of the all X# applications –Reuse of SW from DataGrid and other # projects –Globus as the bottom layer of the middleware –Heterogeneous computer and storage systems –Distributed development and testing of SW 12 partners in applications 14 partners in middleware 15 partners in testbeds

Cracow Grid Workshop, November 5-6, 2001 General rules –Iterative improvement: development, testing on testbed, evaluation, improvement –Modularity –Open source approach –SW well documented –Collaboration with other # projects

Cracow Grid Workshop, November 5-6, 2001 Architecture Team –Activity: Merging of requirements from WP1, WP2, WP3 Specification of the X# architecture (protocols, APIs) Establishing of standard operational procedures Specification of the structure of deliverables –Organization: Representatives of WPs (responsible for integration) Technical Architecture Team

Cracow Grid Workshop, November 5-6, 2001 Layered Architecture Interactive and Data Intensive Applications (WP1)  I nteractive simulation and visualisation of a biomedical system  Flooding crisis team support  Distributed Data Analysis in High Energy Physics  Weather forecast and air pollution modelling Grid Application Programming Environment (WP2)  MPI code debugging and verification  Metrics and benchmarks  Interactive and semiautomatic performance evaluation tools Grid Visualisation Kernel Data Mining New CrossGrid Services (WP3) Globus Middleware Fabric Infrastructure (Testbed WP4) Datagrid GriPhyN... Services HLA  Portals and roaming access  Grid resource management  Grid monitoring  Optimisation of data access

Cracow Grid Workshop, November 5-6, 2001 WP1 Architecture Components 2.2 MPI Verification 2.3 Benchmarks 2.4 Performance analysis 3.1 Portal + Roaming 3.2 Resource Management 3.3 Monitoring 3.4 Optimisation of data access Applications WP3 WP2

Cracow Grid Workshop, November 5-6, 2001 Connection with Datagrid Data Management Replica Manager Replica Catalog interface Replica optimisation CrossGrid optimisation of data access Replica Catalog Automatic data migration & replication

Cracow Grid Workshop, November 5-6, 2001 Connection with Datagrid Workload Management Resource Broker Crossgrid Resource Management Matching job requirements against available resources RB Agent High level Strategy library Logging & Bookkeping Job submission

Cracow Grid Workshop, November 5-6, 2001 Software Integration –HLA - architecture for distributed simulations –CCA - component-based approach –Interoperability with DataGrid

Cracow Grid Workshop, November 5-6, 2001 HLA -Overview –consists of federates (simulations, simulation managers, data collectors, passive viewers) that form federations –is defined of three components: Federation rules proper interaction of simulations in a federation Interface specification defines Run-Time Infrastructure services Object Model Template common framework for HLA object documentation

Cracow Grid Workshop, November 5-6, 2001 HLA - Important Features establishing a common high level simulation architecture separates communication and simulation advanced time management flexible attribute and ownership mechanisms current implementation RTI 1.3 uses UDP multicast communication

Cracow Grid Workshop, November 5-6, 2001 Common Component Architecture –Corba Component Model, Microsoft COM, Java Beans –CCA Forum addresses HPC community –Scientific Interface Definition Language Components and ports Local and remote components –CCA Toolkit (CCAT) framework based on Globus

Cracow Grid Workshop, November 5-6, 2001 Software Integration - Summary Goal:establishing a common high level simulation architecture on Grid for real-time simulations Possible solution:HLA implementation built within component framework over GLOBUS architecture

Cracow Grid Workshop, November 5-6, 2001 S/W Engineering Practices –Developers’ common point of view Introduces work overhead Annoying and boring –But the truth is Using strict operational procedures for Requirements traceability Configuration mgmt Testing... helps to keep project activities on track We know where we are and what we do !

Cracow Grid Workshop, November 5-6, 2001 Reqs. Traceability –The 1 st baseline Defined after the 1 st quarter Base of project work !  Acceptance criteria –Further modifications PRs/CRs mechanism - subsequent baselines (versions). –Identification All WPs deliverables – Refered reqs. baseline  what your work is based on?

Cracow Grid Workshop, November 5-6, 2001 Integration &Testing –WP1, WP2, WP3 Unit tests Need for stubs for other WPs interfaces until they’re available WPs deliverables –WP4 (International Testbed) Integration with DataGrid testbed Integration tests Hw/Sw components Software distribution/installation policy  what are we testing? Test plan Schedule Test logs, feedback distribution – what needs to be changed.

Cracow Grid Workshop, November 5-6, 2001 Configuration Mgmt –Central project repository Version control tool is to be choosen All project ‘official’ artifacts Source of code for WP4 to test –Traceability of configuration items Maintained to upper level components E.g. design  requirements, code  design, etc.

Cracow Grid Workshop, November 5-6, 2001 Conclusion –We must start with an internal report which will present Approaches to the definition of the architecture Coding guidelines Java conventions for naming, comments??? Document publication rules