Page 1GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Resulting Technical Specification

Page 2GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Development of the processing system System design Targets:  Maximum re-use and integration of existing processing / data handling package.  Flexibility to activate one merging technique or the other

Page 3GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Development of the processing system System overview Acceptance of any data sources Stand alone processing system (PC based HW / under GNU/linux oper. syst.) Intermediate individual sensors level 3 products Dissemination of the DDS, PPS, FPS Link with BEAM

Page 4GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Development of the processing system GLOBCOLOUR processing system Main modules:  Pre-processing  Spatial binning  Temporal binning  Merging

Page 5GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST  L3 products at track level  daily L3 products per instrument (temporal binning +data-day)  merged daily L3 products (merging)  8-days and monthly L3 products (temporal binning) Technical approach and discussion Update of the methodology GLOBCOLOUR processing system

Page 6GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Globcolour processor development

Page 7GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Development of the processing system GLOBCOLOUR processing system Pre-processing components:  Application of a cross calibration LUT  Computations of Fully Normalised Water Leaving Radiance  Computation of a common K d (490)

Page 8GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Development of the processing system GLOBCOLOUR processing system Spatial binning Application of a flux-conservative resampling algorithm (using the Sutherland- Hodgeman area Clipping technique) L2 grid L3 grid

Page 9GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Development of the processing system GLOBCOLOUR processing system Spatial binning Main software component developed:  Product reader for all types of Level-2 input products; MERIS L2 (Envisat PDS format), SeaWiFS and MODIS /Aqua Ocean Colour Level 2 in (HDF-EOS), POLDER Ocean Colour Level 2 (HDF format)  Resampling Component  DDS Level-2/Level-3 Extraction Component  DDS Level-2/Level 3 granule netCDF writer  Product writer for the spatially binned, intermediate level-3 netCDF products

Page 10GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Development of the processing system GLOBCOLOUR processing system Merging Several software components for merging are currently under IDL breadboard implementation:  One or more implementations for the merging method to be used, e.g. a. Weighted Averaging b. Optimal Interpolation c. Blended Analysis d.GMS  An ocean colour retrieval processor starting from merged sensor spectra

Page 11GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Development of the processing system GLOBCOLOUR processing system Temporal binning The core software components have been developed:  A component performing the weighted averaging  DDS Level-3 extraction component  DDS Level-3 granule netCDF writer  Level-3 data product writer using netCDF format Components not yet consolidated:  A component used for computing the error statistics  Data-day algorithm for all instruments (breadboard under validation)

Page 12GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Development of the processing system Input Products Overview Uncompressed data volume

Page 13GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Development of the processing system Output Products Overview

Page 14GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Development of the processing system Output parameters (1/3)

Page 15GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Development of the processing system Output parameters (2/3)

Page 16GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Development of the processing system Output parameters (3/3)

Page 17GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Development of the processing system ISIN grid main characteristics

Page 18GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Development of the processing system Output products variables - definition (1/2)

Page 19GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Development of the processing system Output products variables – storage (2/2) + 15 attributes: scaling factors, names, unit, etc…

Page 20GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Development of the processing system Format modification since TS 1.1 (globCOLOUR processor v1.0) Variables are no more scaled due to hot points occurrence hot points The hot points are due to a high coding quantisation error at very small weight bins Impact on final product size has been assessed and taken into account in the re-estimation of the required resources

Page 21GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Development of the processing system DDS/PPS/FPS products – overall list (1/2)

Page 22GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Development of the processing system DDS/PPS/FPS products – content (2/2)

Page 23GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Production of the Full Product Set Processing requirements  one month of data from the 4 sensors processed in 5 days  ten years of data processed in 6 months  NRT L3 processed in less than 24 hours Preliminary data flow requirements and system sizing has been re-assessed using the globCOLOUR processor v1.0 installed on the APADIC system at ACRI-ST

Page 24GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Production of the Full Product Set APADIC system in ACRI-ST

Page 25GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Production of the Full Product Set Resources requirements analysis

Page 26GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Production of the Full Product Set Resources requirements analysis

Page 27GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Production of the Full Product Set Resources requirements analysis

Page 28GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Production of the Full Product Set Resources requirements analysis

Page 29GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Production of the Full Product Set Resources requirements analysis

Page 30GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Production of the Full Product Set Resources requirements analysis

Page 31GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Production of the Full Product Set Resources requirements analysis

Page 32GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Diagnostic Data Set DDS objectives The processing system creates DDS granules for a minimum of 20 predefined diagnostic sites. DDS granule are extracted from Level-2 input products and Level-3 output products A DDS granule covers approximately 100km x 100km and comprises all layers of information of the input or output products. The granules are provided in the netCDF format following the globCOLOUR DDS format (following CF 1.0 rules)..

Page 33GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Diagnostic Data Set DDS important characteristics DDS are defined by center lon-lat and temporal period. Close DDS lead to almost identical DDS L2 products. DDS bins are computed using a nearest interpolation approach. One pixel of the L2 track is projected on a single DDS bin: holes exist in the built DDS map (see next slides). Identified by flags. DDS from L2 input products: PC 1 km DDS from L3 output products: ISIN 4.63 km (automatic reprojection on PC map performed by DDS tools)

Page 34GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Diagnostic Data Set DDS holes MERISMODIS

Page 35GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Diagnostic Data Set DDS holes - flags

Page 36GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Diagnostic Data Set Diagnostic Sites Two purposes  Sensors characterisation against in situ data  Cross-characterisation for specific biogeographical areas The DDS location and spatial extent are selected from the existing validation sites, e.g. IOCCG/SIMBIOS DDS, ESA validation sites, using the same protocol of selection as the one used in SIMBIOS..

Page 37GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Diagnostic Data Set Diagnostic Sites First pre-selection (1/2).

Page 38GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Diagnostic Data Set Diagnostic Sites First pre-selection (2/2) GlobCOLOUR Diagnostic Data Set locations

Page 39GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Technical approach and discussion Diagnostic Data Set Already generated DDS L2 products

Page 40GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Globcolour processor status Version 1.0: used at ACRI and at UoP to generate DDS regression tests Version 2.0: to be released 1 st of September