Stefano Nativi 1 Co-authors Paolo Mazzetti 1, Valerio Angelini 1 Marco Verlato 2, Mirco Mazzucato 2 CYCLOPS: an EGEE SSA Implementation of Geosciences services on Grid platform for GMES applications 1 CNR - IMAA 2 INFN

2 Outline CYCLOPS Overview – Rationale – Objectives – Main Activities Architectural Framework Spatial Data Infrastructure Services implementation

3 CYber-Infrastructure for CiviL protection Operative ProcedureS (CYCLOPS) Specific Support Action of EGEE Duration: 24 months (01/06/2006 – 31/05/2008) Web site: Contact persons: – (Project Manager) – (Technical Manager) Keywords: GMES, GRID, Geospatial information, Civil Protection Collaborations with: – PREVIEW, Risk EOS, RISK-AWARE, BOSS4GMES, EGEE, DEGREE

4 Participants Civil Protection Agencies – CP-CH (Civil Protection of Chania Prefecture) – DDSC (Direction de la Défense et de la Sécurité Civiles) – DPC (Dipartimento della Protezione Civile) – SNBPC (Serviço Nacional de Bombeiros e Protecção Civil) Scientific/Technological partners – EMA (Ecole des Mines d' Ales ) – IMAA-CNR (Istituto di Metodologie per lAnalisi Ambientale del Consiglio Nazionale delle Ricerche) – INFN (Istituto Nazionale di Fisica Nucleare) – TEI-CR (Technological Educational Institute of Crete)

5 Rationale Civil Protection and GMES applications/systems have specific requirements: – to access infrastructure, run models and search information in a real-time (RT) or near-real-time (NRT) way privileging time of response instead of accuracy – to control sensors networks and acquisition systems and modify their acquisition strategy and processing chain – to share geospatial information that has complex characteristics: Huge amounts of remotely-sensed observations, which are multidimensional and frequently updated – To formalise the knowledge required to analyse data and provide decision-makers with effective information – To implement the strict data policy and the security requirements typical of dual systems (civil/military);

6 Why to use Grid Platform for GMES applications? 1. Provide physical resources to support application workloads a. Provide the computational power for working out simulation tasks or tasks for critical missions (see job management) b. Provide near real time access to data resources (see data management) 2. Virtual Organization management a. Grid information system b. Resource cataloguing and sharing 3. Provide a security infrastructure 4. Provide a quality of service agreement & monitoring infrastructure 5. Provide an infrastructure to distribute VO policy management on its nodes

7 Objective & Research Strategies Conceive a complete Grid-based platform supporting Civil Protection/GMES applications – The CYCLOPS platform Many open issues must be addressed – Integration & Interoperability (with EGEE platform) Implement Spatial Data Infrastructures (SDI) services in the EGEE platform Integrate sensor networks and acquisition systems in the EGEE platform – Define specific services for GMES/CP applications Define advanced services (knowledge-based services, Quality-of-Service management, etc.) Define Data Policy and Security services Define generic Business Logic, Presentation and Fruition services

8 CYCLOPS Main Activities Definition of research and innovation strategies – Analysis of CP systems – EGEE Request for Enhancements – Research strategies for enabling CP applications on Grid infrastructure – Guidelines for CP innovation towards the adoption of Grid technologies Cross-dissemination between GRID (EGEE) community and GMES (Civil Protection) community. – Seminars, workshops, tutorials directed to CP personnel – Reports to EGEE Working Groups

9 Standardization Communities GRID community – test and validate, the middleware re-engineered by EGEE and EGEE-2 in compliance with the current open standards (GGF, OMII Europe) Geospatial Information community – INSPIRE (The INfrastructure for SPatial InfoRmation in Europe ) – GEOSS (Global Earth Observation System of Systems) – OGC/ISO (Open Geospatial Consortium) Risk and Crisis Management Sensor Web Enablement Earth Observation OWS: WCS, WFS, CS/W, CTS, WPS, … Civil Protection community – GMES (Global Monitoring for Environment and Security)

10 Selected use case: Flood Forecast Involves heavily meteorology and hydrological modelling Involves many sensors (floods gauges, rain radars, meteo sats) Linked to GMES flash flood anticipation service

11 Selected use case : Large forest fire propagation Propagation models used on the field are simplistic due to computational limitations More sophisticated models require heavy computations, large number of field and meteo parameters French research project PAREFEU, link with GMES service PREVIEW After Sero-Guillaume et al.

12 Architectural Framework

13 Communities Interoperability GMES & CP Communities Geospatial Community GRID Community Strategic Applications Strategic Applications Geospatial Information services Geospatial Information services Resources Provision Resources Provision Semantic Level

14 Spatial Data Infrastructure tiers WFS WCS WMS Access and Download services Discovery services based on ISO profiles CS-W WMS Thematic Portals View services View services Presentation services Resources Processing Services

15 CYCLOPS Platform CYCLOPS Infrastructure Spatial Data Infrastructure Services CYCLOPS Architectural Framework Processing Systems Infrastructure Data Systems GRID Platform (EGEE) Security Infrastructure Real Time and Near Real Time Applications for Civil Protection (Data integration, high-performance computing and distributed environment for simulations) Real Time and Near Real Time Applications for Civil Protection (Data integration, high-performance computing and distributed environment for simulations) Interoperability Platform Sensor Element Services Advanced Grid Services Business logic Services Presentation and Fruition Services Environmental Monitoring Resource Infrastructure Geospatial Resources Services

16 Spatial Data Infrastructure Services implementation

17 Implementing OWS on the top of a Grid Platform Web over Grid Grid-enabling OWS – Build OWS implementations which make use of Grid capabilities Grid over Web Gridifying OWS – Expose OWS as Grid computational elements (CE) which run native calls to external WS

18 Grid-enabling OWS netCDF HDF GRIB Others SE CE OWS implementation Software (published via GRIS) SRM interface Resource Broker/ WM proxy From OWS request to JDL Portal/ submissi on tool on Grid User Interface DLS (Data Location Service) based on LFC

19 Geospatial Information GRID platform ---SE HostFile Block DatasetFile Coverage/Layer/Map--- Logical Aggregation Level Host name LFN for a file block netCDF HDF GRIB Grid-enabling OWS Data Aggregation levels (semantic levels)

20 Grid-enabling OWS: OWS function Vs. Grid job (OWS Server) Service Provider Service Consumer Service Register 1. OWS server URL 2. OWS.function request 3. OWS.function response document 1. OWS server URL LFN for file block 2. OWS.function request Grid job for a. CEs publishing the required software tag; b. CEs close to the SEs containing a LFN replica 3. OWS.function response document URL to get job status & job output document netCDF HDF GRIB

21 GRID-enabling OWS OWS Request DAG Job CE near file-based Dataset

22 Grid-enabling WCS A prototype of Grid-enabled WCS on top of EGEE Platform Implementation of basic WCS functionalities like subsetting, resampling and interpolation using GRID capabilities Expected advantages: – Grid support of file replicas – Use of mobile code instead of mobile data – High performances working on large datasets – Scalability

23 Traditional WCS WCS interface Data Subsetting Resampling DataEncoding Data Encoding RequestResponse

24 GRID WCS WCS interface Data Subsetting Resampling DataEncoding Data Encoding EGEE Grid Interface Request CE Async. response SE EGEE Grid Interface CE EGEE Grid Interface CE EGEE Grid Interface Workload Manager S S RRR D R R R D WCS Interface Result Result URL

25 GRID WCS : Technologies EGEE Grid Interface CE SE EGEE Grid Interface CE EGEE Grid Interface CE EGEE Grid Interface Workload Manager WCS Interface WEB Application Layer (Java) EGEE GRID Computing Layer (Java) Request Async. response ResultHTTP/SOAP S R R R D JDL Java Serialization S RRR D

26 Gridifying OWS Run application software which calls OWS external services – Implement a Grid application which makes use of OWS service chaining – Possible use of all Grid job types: collection, DAGs, Parametrics, MPI, etc. Minimal case: Run an existing OWS function as a Grid job netCDF HDF GRIB + + Grid Job

27 Gridifying OWS SE CE Resource Broker/ WM proxy WSDL netCDF HDF GRIB Others... Native call OWS

28 Virtualize Sensors Two approaches: – A new Grid sensor element Instrument Element (like GridCC?) – Make use of OGC Sensor Web Enablement services CYCLOPS Infrastructure Spatial Data Infrastructure Services Processing Systems Infrastructure Data Systems GRID Platform (EGEE) Sensor Element Services Advanced Grid Services Environmental Monitoring Resource Infrastructure Geospatial Resources Services

29 Thank you for your attention