WP 9.4 Use Case (ESA-ESRIN, IPSL, KNMI) Annalisa Terracina, ESA-ESRIN Earth Observation work package (WP9)

Earth Observation use case The processing, archiving and validation of GOME ozone profile products using two different algorithms and ground based measurements for validation form the basis of this use case. This use case is an example of common EO data usage and EO science: the 'raw' data is processed into products, using different algorithms. The outcome is then validated using ground-based measurements. One aspect to tackle is to find the proper GOME pixels that coincide with the ground based measurements, for this we are going to use a grid enabled database, to store geo- and time- references of GOME files/pixels Goal: do this for one year of GOME data

Earth Observation DEMO Raw satellite data from the GOME instrument Earth Observation DEMO for PM 24 LIDAR data Processing of raw GOME data to ozone profiles With OPERA and NNO Validate GOME ozone profiles With Ground Based measurements DataGRID Visualization

GOME Use Case + KNMI ESA ESA Science Application End User IPSL 4724 files = 66 Gb 9,448,000 files = 108 Gb KNMI L1 L2 ESA ESA RAW L1 L1 L2 Science Application End User L2 + L3 IPSL VAL L2 VAL Regulated Access to Grid processing power Secure access to Grid-registered high-volume data storage

Scenario overview: PM24 ESA will provide GOME level 1 data on a GRID storage element (SE). The meta information is stored in a GRID accessible database, Spitfire. This data and meta data is used by KNMI to produce Ozone profile data using the Opera S/W, by running it on a Computing Element (CE). The profile data (level 2) is stored on a GRID SE. The profile meta data is stored in a database accessible on the GRID via Spitfire. ESA will also use the level 1 data and meta data by retrieving level 1 files from a SE, and producing Ozone profiles. A Neural Network algorithm will do the processing. ESA will store the profile data and meta data on the GRID via a SE and Spitfire. The IPSL application will perform a validation of the produced ozone profiles. The profiles are validated against in situ LIDAR ozone profile measurements. IPSL will retrieve profile data (OPERA and NNO) from the GRID, querying the provided meta data to find data which coincides with their measurements. The LIDAR data will be stored on a SE on the GRID.

File numbers for one year of GOME data : Number of files (to be stored and replicated): File Size Level 1: 4,724 15 Mb NNO: 9,448,000 10 kb Opera : 12 kb Lidar 12 2.5 Mb Total: 18,900,736 267 Gbyte Gome has a data set of 5 years Gome is relatively small (in both size and number of files)

Use case flow for profile processing:

Questions: Is the amount of files a problem? (for replication and meta data storage) Does the use case contain enough detail for middleware? Other questions?