Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS Dave Poulter Ocean Observing and Climate Group and Laboratory for Satellite Oceanography National Oceanography Centre, Southampton UK Project Logo

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS Dave Poulter Status and Applications of the HR-DDS Project Logo

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS The HR-DDS concept Project Logo The HR-DDS system is a distributed archive and analysis system for subsets of GHRSST-PP and other data. Satellite SST ObservationModel output and estimationsIn situ obervations

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS The HR-DDS concept Project Logo Data are ingested in real time, re-gridded to a common 0.01 degree grid and reproduces in standard format over by 2 degree areas.

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS The HR-DDS concept Project Logo Allows for easy intercomparison of data interactively online

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS The HR-DDS concept Project Logo Highly configurable and expandable analysis system

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS Why do we need and HR-DDS? Project Logo

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS Status Overview - A years progress! Project Logo A new look GHRSST ‘style’ website Inclusion of L2Pcore and non GHRSST datasets Enhanced statistical analysis functions New and far more powerful graphics engine Redesigned processing backend - now DISTRUBUTED. Prototype climatology datasets Semantic content awareness Prototype real time data metrics Prototype direct data downloads

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS New Distributed Architecture The system is now distributed: There are many possible modes - this is a single computer HR-DDS system Project Logo SQL Register & Stats DB FTP DODS THREADS HTTP Processing System

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS New Distributed Architecture This is the currently system running at NOCS Project Logo SQL Register & Stats DB FTP DODS THREADS HTTP Processing System SQL Register & Stats DB Processing System Backup DB Backup Archive

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS New Distributed Architecture This is planned system architecture at NOCS Project Logo FTP DODS THREADS HTTP MODIS Processing System Backup Processing System Backup DB Register & Stats DB Re - Processing System Primary Processing System Backup Archive

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS New Graphics Engine The new graphics engine allows us, for the first time, to have the following: Project Logo Error bars Readable dates Line styles Font styles

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS New Graphics Engine Significantly, the new engine reads the netCDF HR-DDS files and generates maps on the fly. This means that there is no need for the generation of quick-look image files - the webpage is entirely dynamic. The major advantage is that the page can draw other fields than SST. Project Logo

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS Enhanced Statistical Analysis The HR-DDS can now plot apply bias estimates online Project Logo

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS Enhanced Statistical Analysis The HR-DDS website also records ‘centre only’ values Project Logo

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS Enhanced Statistical Analysis And also ‘best quality only’ values - based on native quality flags Project Logo

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS A New Look HR-DDS Website Project Logo

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS L2P Core and Non Medspiration Data: Ingesting L2PCore AMSRE data from REMSS and JAXA Ingesting FOAM 1 degree and 1/9 degree 5m Temperature Ingesting AVHRR 18 LAC from NAVO Ingesting AVHRR 17 and 18 GAC from NAVO Project Logo

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS Climatologies Included: The system now includes the Pathfinder 4km Pentad climatology and the Reynolds 1 degree daily climatology: Project Logo

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS Semantic Content: The new design contains an interpretive layer in the backend, which ensures that ‘user’ concepts such as ‘sea surface temperture’ can equate to variable names such as ‘analysed_sst’, ‘foundation_sst’ and ‘sea_water_potential_temperature’. Similarly - the database is aware of SST type (foundation, skin etc), SST origin (model, satellite, in situ etc) and depth (skin, 1m, blended etc) Project Logo

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS Analysis on demand: The following plots were generated by our prototype system, and will soon be available on demand. The exact specification of these plots can be set up specifically for any user - proving a user driven metric. Project Logo

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS Analysis on demand: Standard deviation of night time SEVIRI compared to AATSR match ups. Centre values only, all quality levels. No bias correction applied and match ups +/- 3 hours only. Project Logo This took 25 seconds!

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS Analysis on demand: Bias of OSTIA to AATSR - same conditions. Blue = OSTIA Cool Project Logo 15 seconds!

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS Analysis on demand: Number of AVHRR 18 L match ups - same conditions. Project Logo 38 seconds

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS Analysis on demand: I just thought of this at lunch!: This system could be used to return the URL of L2P files where diurnal warming is likely have occurred - in rank order: RankNumber of SitesURL 138http:// 225http:// Project Logo

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS Quality Control Services: Project Logo Provided format specifications and corrections for UK Remote Sensing and Data Analysis System for AVHRR LAC products The HR-DDS system has provided quality control and format specification to JAXA for AMSR-E L2PCore data

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS The Future: The HR-DDS has secured funding from the European MyOceans project. Project Logo We are secure for at least end Money is set aside for operation maintenance, reprocessing and continual development. The HR-DDS is moving from the University of Southampton to the UK Strategic Research Division There is a clear remit with the project for user driven metrics, for example ‘traffic light’ performance indicators.

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS The Future II: There have been good communications with the ESA GLOBColour project Project Logo A merged interface has been defined - GHRSST users will be able to retrieve OC data from the GLOBColour DDS using the GHRSST HR-DDS system. The GHRSST HR-DDS has mirrored all GLOBColour HR-DDS sites. Similar experimental interfaces have been defined for the MDB.

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS What can we do for you? You are the HR-DDS users! We are now in a position to offer specific services to specific users - G8 priority #7 ‘user led applications’. We can provide an alert to you if you want to know when AVHRR17 is more than a degree warmer that SEVIRI within a 1 hour match up criteria. We can ingest data from YOUR buoy array….. We can provide daily maps of coral bleaching conditions…… Project Logo

Meeting Description 8th GHRSST-PP Science Team MeetingMelbourne, Victoria - 14th - 18th May 2007 Status and Application of the HR-DDS Acknowledgements! Fred Wimmer and Chris Jefferys at NOCS have helped with development. Most science team members have helped at some point or another! Project Logo