CNR experience within CIOP: porting the SBAS-InSAR software into the cloud F. Casu, I. Zinno, S. Elefante, P. Imperatore, M. Manunta CNR-IREA Via Diocleziano 328, 80124, Napoli

CNR experience within CIOP: porting the SBAS-InSAR software into the Outlook The SBAS-InSAR algorithm: – scope and main application Why the cloud? Porting SBAS-InSAR into cloud environment – Sequential algorithm – Barely parallel solution – CIOP adaptation – Performances Lesson learnt and future expectation

CNR experience within CIOP: porting the SBAS-InSAR software into the SBAS-InSAR Processing ~150 Satellite raw (L0) data per scene: 40GB Sequential Processing Time: 300h CNR Earthquakes Volcanoes Oil & Gas Water Resources

CNR experience within CIOP: porting the SBAS-InSAR software into the SBAS-InSAR Application L’Aquila earthquake Civil Protection Risk Management Insurances

CNR experience within CIOP: porting the SBAS-InSAR software into the SBAS-InSAR Application Mt. Etna volcano ERS-1/2 and ENVISAT data ( ) Eruption Civil Protection Risk Management Natural Hazards

CNR experience within CIOP: porting the SBAS-InSAR software into the SBAS-InSAR Application ERS-1/2 Descending data ( ) Los Angeles >1<- 1 cm/year 0 Oil & Gas Field Oil extraction

CNR experience within CIOP: porting the SBAS-InSAR software into the SBAS-InSAR Application ERS-1/2 Descending data ( ) Los Angeles >1<- 1 cm/year 0 Agriculture Sustainable and social development Water Resources SBAS-InSAR Application

CNR experience within CIOP: porting the SBAS-InSAR software into the Building and infrastructures analysis >1<- 1 cm/year 0 Damage assessment Insurances ESA - ESRIN YOU ARE HERE (and safe)! SBAS-InSAR Application Rome

CNR experience within CIOP: porting the SBAS-InSAR software into the Past, present and future SAR Satellite Constellations Time

CNR experience within CIOP: porting the SBAS-InSAR software into the Too many data Too many users Why the cloud?

CNR experience within CIOP: porting the SBAS-InSAR software into the Why the cloud? Wide application scenarios Process large archived (ERS, ENVISAT, …) or forthcoming (CSK, TSX, Sentinel-1) amount of SAR data High resource demand during emergency phases Processing time must be “reasonable” Make the processed data/achieved results/processing tools available to the community: implement the Supersites concept

CNR experience within CIOP: porting the SBAS-InSAR software into the CIOP Application Platform Data & Catalogue User (CNR) Cloud Controller API OCCI Web Interface Sandbox private public

CNR experience within CIOP: porting the SBAS-InSAR software into the User (CNR) Web Interface Sandbox CIOP Application Platform

CNR experience within CIOP: porting the SBAS-InSAR software into the Sequential SBAS Flow Diagram SLC Scaling selection SLC cut DEM to SAR conversion Range File computation DEM Focusing Master Focusing Parameters Computation Data Deformatting OrbitsSAR Data Shift Computation Common Area 2nd step Registration Registration RefinementSLC Scaling SLC Registration Interferometric Pairs Selection DEM Shift Calculation Interferogram Computation Displacement time series

CNR experience within CIOP: porting the SBAS-InSAR software into the Sequential test case 7 ASAR-ENVISAT acquisitions 1 core 8 GB RAM 500GB of storage (NFS)

CNR experience within CIOP: porting the SBAS-InSAR software into the Sequential SBAS Flow Diagram SLC Scaling selection SLC cut DEM to SAR conversion Range File computation DEM Focusing Master Focusing Parameters Computation Data Deformatting OrbitsSAR Data Shift Computation Common Area 2nd step Registration Registration RefinementSLC Scaling SLC Registration Interferometric Pairs Selection DEM Shift Calculation Interferogram Computation Displacement time series 5 min 33 min 190 min 3 min 0.3 min N/A Negligible 75 min 70 min 66 min 5 min 111 min 7.3 min 24 min 56 min N/A ~14.8 hours 247 min

CNR experience within CIOP: porting the SBAS-InSAR software into the Sequential SBAS Flow Diagram SLC Scaling selection SLC cut DEM to SAR conversion Range File computation DEM Focusing Master Focusing Parameters Computation Data Deformatting OrbitsSAR Data Shift Computation Common Area 2nd step Registration Registration RefinementSLC Scaling SLC Registration Interferometric Pairs Selection DEM Shift Calculation Interferogram Computation Displacement time series Orbits 33 min 190 min 3 min 0.3 min N/A Negligible N/A 75 min 70 min 66 min 5 min 111 min 7.3 min 24 min 56 min 5 min 247 min ~14.8 hours

CNR experience within CIOP: porting the SBAS-InSAR software into the Barely parallel test case 7 ASAR-ENVISAT acquisitions 2 cores 8 GB RAM per core 500GB of storage (NFS)

CNR experience within CIOP: porting the SBAS-InSAR software into the Sequential SBAS Flow Diagram SLC Scaling selection SLC cut DEM to SAR conversion Range File computation DEM Focusing Master Focusing Parameters Computation Data Deformatting OrbitsSAR Data Shift Computation Common Area 2nd step Registration Registration Refinement SLC Scaling SLC Registration Interferometric Pairs Selection DEM Shift Calculation Interferogram Computation Displacement time series Orbits 33 min 190 min 3 min 0.3 min N/A Negligible N/A 75 min 70 min 66 min 5 min 111 min 7.3 min 24 min 56 min 5 min 247 min ~14.8 hours

CNR experience within CIOP: porting the SBAS-InSAR software into the ~10 hours Parallel SBAS Flow Diagram SLC Scaling selection SLC cut DEM to SAR conversion Range File computation DEM Focusing Master Focusing Parameters Computation Data Deformatting OrbitsSAR Data Shift Computation Common Area 2nd step Registration Registration Refinement SLC Scaling SLC Registration Interferometric Pairs Selection DEM Shift Calculation Interferogram Computation Displacement time series Orbits 33 min 190 min 3 min 0.3 min N/A Negligible N/A 5 min75 min 70 min 66 min 5 min 111 min 7.3 min 24 min 56 min 247 min 2.5 min 100 min 1.8 min 39 min 34 min 56 min 3.8 min 30 min 177 min

CNR experience within CIOP: porting the SBAS-InSAR software into the Pros and Cons Scalable (on Images & Interferograms) Easy to implement and debug No job scheduling and balancing Not fully portable

CNR experience within CIOP: porting the SBAS-InSAR software into the SBAS Flow Diagram within CIOP SLC cutDEM to SAR conversion Range File computation DEM Focusing Master Focusing Parameters Computation Data Deformatting OrbitsSAR Data Shift Computation Common Area 2nd step Registration Registration Refinement SLC Scaling SLC RegistrationInterferometric Pairs Selection Collec t

CNR experience within CIOP: porting the SBAS-InSAR software into the Interferogram Selection Coherent point Selection Tie Point Refinement Unwrapping Time Series Generation Residual Phase Computation Orbital Ramp Estimation Filtered Interferograms Unwrapping Tie Point Refinement Time Series Generation Residual Phase Computation Atmospheric Filtering Orbit Correction DEM Shift Calculation Interferogram Computation Collec t SBAS Flow Diagram within CIOP Collec t yes no End

CNR experience within CIOP: porting the SBAS-InSAR software into the IREA cluster vs. CIOP Sandbox comparison 7 ASAR-ENVISAT acquisitions 2 cores 8 GB RAM per core 500GB of storage IREACIOP ~10 hours~13 hours

CNR experience within CIOP: porting the SBAS-InSAR software into the IREACIOP Negligible Focusing Master Focusing Parameters Computation Data Deformatting OrbitsSAR Data Shift Computation Common Area SLC Scaling Collec t IREA vs. CIOP Sandbox Comparison

CNR experience within CIOP: porting the SBAS-InSAR software into the Focusing Master Focusing Parameters Computation Data Deformatting OrbitsSAR Data Shift Computation Common Area SLC Scaling Collec t SLC cutDEM to SAR conversion Range File computation DEM 2nd step Registration Registration Refinement SLC RegistrationInterferometric Pairs Selection Collec t IREA vs. CIOP Sandbox Comparison

CNR experience within CIOP: porting the SBAS-InSAR software into the SLC cutDEM to SAR conversion Range File computation DEM 2nd step Registration Registration Refinement SLC RegistrationInterferometric Pairs Selection Collec t IREA vs. CIOP Sandbox Comparison IREACIOP

CNR experience within CIOP: porting the SBAS-InSAR software into the Unwrapping Preparation Unwrapping Time Series Generation Orbital Ramp Estimation Final processing and Atmospheric Filtering Orbit Correction DEM Shift Calculation Interferogram Computation Collec t SBAS Flow Diagram within CIOP Collec t yes no Collec t IREACIOP

CNR experience within CIOP: porting the SBAS-InSAR software into the >1<-1 cm/year Analyzed time interval: Real case study (Benchmark) IREACIOP ~24 hours~48 hours 55 ASAR-ENVISAT 8 cores 8 GB RAM per core 1TB of storage Napoli Bay area

CNR experience within CIOP: porting the SBAS-InSAR software into the CIOP Pros & Cons Scalable (on Images & Interferograms) Relatively easy to port scientific algorithms, after a preparatory learning phase on the environment Collect steps represent a bottle neck! (large data transfer overhead) Portable, but built on Open Nebula which could be not supported by the cloud provider (need of additional driver)

CNR experience within CIOP: porting the SBAS-InSAR software into the Final considerations CIOP has been conceived to be the most general and portable as possible but, at the present stage/configuration, seems to be not the optimal solution for an application which is not characterized by independent jobs For SBAS purposes, different environment solution (cloud resources as grid computing, single virtual workstation on the cloud) could be explored

CNR experience within CIOP: porting the SBAS-InSAR software into the Final considerations The presented activities have been carried out within the Helix Nebula framework, which aims at providing a federated European science cloud The ESA SSEP flagship, aimed at creating a common scientific ecosystem for sharing data, querying tools, processors, and added value results, depicts an ambitious goal, which found in Helix Nebula its natural growth environment The large debate within Helix Nebula for the generation of a unified driver for federated cloud implementation should be carefully taken into account by ESA