Calibration Event Logging GSICS meeting, March 2013, Williamsburg USA Rob Roebeling and Sebastien Wagner EUMETSAT

Outline Calibration Events Working Group (CE-WG) achievements 2011-2012 Proposed calibration event logging system Define a high level procedure for logging calibration events Encourage the use common standards Encourage populating the calibration event databases with historic events Open up the database Plans CE-WG 2013-2014 Work on action set by CGMS to be reported at next CGMS (July 2013) Work in the CE-WG’s work plan 2013-2014

Achievements (2011-2012)

CE-WG Achievements 2011-2012 Initiation of the CE-WG Assessment of Eumetsat’s event logging systems (UNS and CHART) Collection of User Requirements (internal and external presentations) Discussion of System Requirements (internal) High level concept of the Calibration Events Logging System Presentation of a working paper on current status at CGMS-40 CGMS action 40.15 (before July 2013) NOAA and EUMETSAT are invited to present a consensus concept and realisation of a calibrations events logging system with emphasis on issues and lessons learned. Due date: CGMS-41.

High Level design Calibration Event Logging System

General principles The proposed Calibration Event Logging System should : log all events impacting radiometric and geometric data quality:  Platform level  Satellite level  Data processing level use common Standards and Procedures across missions and instruments; allow for portability across space agencies; comprise a database historic events; allow users to assess and visualize the events database. Note no duplication of existing systems (easier development + maintenance) but direct link to them + additional info when required.

General concept – Uniform Calibration Event Logging System Web Interface Mission Calibration Events Database Users Mission Events Database

Virtual hierarchical design – Uniform Calibration Event Logging System Mission Events Database

Main Categories Mission Information General information on the platform, instruments and sensors operated in the mission that is relevant to all users of the satellite data. Irregular Events Events at satellite level that are not occurring systematically and that impact the radiometric or geometric quality of the observations due to instruments calibrations, manoeuvres or miscellaneous. Processing Events Updates of the processing algorithms (onboard or ground seg.) or reprocessing events (ground seg.) that modify the radiometric or geometric quality of the observations. Data Outages Events that triggered the temporary or definitive end of the data collection.

Events Types as logged in existing systems UNS (Eumetsat): instrument-calibration, data interrupted, instrument-maintenance, ground- segment-maintenance, instrument-gain-change, eclipse-season, spacecraft-anomaly, ground- segment-anomaly, in-plane-manoeuvre, product-change, .. ASCAT: Manoeuvres; heat refuse , instr. switch off, software probl., processor update, sun eclipse, transponder event,…. GOME: missing scan(s), degraded MDRs (instrument), saturated scan(s), mode changes, data gap, manoeuvre events, tank heater on/off, NEDT-PEAK event, channel swap, … METEOSAT: ice decontamination, calibration coefficient update, gain changes, update calibration processor, instr. switch off, … METOP: Antenna correction, channel degradation, update VIS calibration coefficients, .. VIIRS: stray light, striping issues, roll manoeuvre, lunar calibration, warm up, cool down, anomaly, accuracy switch, calibration Blackbody Warm-up Cool-down, door open… POES: scan motor, optics, antenna, electronics, channel saturation, data outage, radiant cooler, calibrator, …. ASCAT - Manoeuvres (in-plane [shorter data outage] + out-of plane [longer data outage]). Reported as well as the resulting data outage. - heat refuse, causing exceedence of the noise limit. Causes a data outage. - Switch off of the instrument (on-board software problem) - Sun eclipsed by the Moon. But impact on data is not known. - Processor update - Events coming from product monitoring (such as the power gain product) - Calibration campaigns. Causes 1/ data outage during the calibration campaign update of the calibration (2 months, twice a day causing 20 minutes data-outage + poor quality) Events on transponders if happening when the update on the calibration is performed. GEO - ice decontamination for IR bands - update of the calibration coefficients in the image headers for the VIS/NIR bands - update of the calibration processor (VIS/NIR) SSCC - gain changes - instrument switch off METOP Antenna correction (AMSU) Channel degradation irregular update of visible calibration parameters GOME missing scan(s) degraded MDRs (instrument) scan(s) with old dark correction applied saturated scan(s) Mode changes Data gap Manoeuvre events TDIF event Tank heater on/off Operator events NEDT-PEAK event Life Limiting Items Channel Swap Anomaly Message USN instrument-calibration Data interrupted general-announcement instrument-maintenance ground-segment-maintenance instrument-gain-change eclipse-season ground-segment-anomaly in-plane-manoeuvre spacecraft-anomaly product-change new product VIIRS Stray light Striping issues Roll manoeuvre Lunar calibration Warm up Cool down Anomaly Door open Table update Calibration Blackbody Warm-up Cool-down Accuracy switch

Events Database – Events Types Events Types List Operational status changes (e.g. Switch to operational..) Command chain anomalies (e.g. ..) Operating mode changes (e.g. Switch to rapid scan or full disk mode, Clock errors, ...) Platform alteration Platform damages Instrument maintenance (e.g. Stray light correction, Ice decontamination, ...) Instrument alteration (e.g. Detector temperature changes, Solar diffuser changes, ..) Instrument damage (e.g. CCDs failure, Mechanical anomalies, Switch to redundant systems, ..) Detector configuration (e.g. Dark current, Space view position, ...) The CE-WG will propose to define: -> an EventsType List, comprising standard calibration events names -> a List of mandatory Attributes to be provided per EventsType

Standards and Procedures Common Standards and Procedures are required to improve traceability and uniformity cross space agencies. This means: Design data model following inter-national standards (ISO, OGC, …); Attune to standards used in other CGMS/GSICS/WMO programs; Use uniform (mandatory) naming conventions and attributes for each EventList Obj.; Use uniform (mandatory) naming conventions and attributes for the EventType Obj.; (e.g. instrument, date of change, link to online resource, responsible agency, ..) Provide links to documents with detailed and quantitative Event information ; Provide the databases in common data formats (e.g. NetCDF, ..).

Populating the Database Many of the existing databases of calibration satellite events  don’t keep all required information,  don’t contain events from historic missions. Thus: additional information needs to be collected to complete the database. historic events need to be mined, and knowledgeable users need to be consulted. space agencies need to assess what historic events are and can be made available. Some events are confidential, other haven’t been logged (especially older events)). Note Populating the database with historic events is a requirement of the climate community, but has no priority for the CGMS and GSICS!

Opening up the Databases Suggestions for reporting interface (e.g. web based data base); Open up the databases through a Graphical User Interface (GUI); GUI shall be high-level enabling intuitive event log searching; GUI shall be tested on a set of predetermined queries; GUI shall include manners to display groups of events graphically (e.g. outages).

Plans (2013-2014)

Actions required by CGMS till meeting in 2013 CGMS and GSICS partners to review the draft Calibration Events Logging System; GSCIS and CGMS partners to discuss and propose the common standards to adopt; Space Agencies to define the “high-level” and the “specific” system requirements; Space Agencies to test system concept through small pilot projects; Space Agencies to identify Climate Users that can provide information on historic calibration events.

CE-WG Work Plan - 2013-2014 Initiate an internal project (ECPC): to further assess requirements; to explore how to build upon existing systems; to consolidate user and system requirements; to set up a pilot calibration event logging system Collaborate NOAA, GSICS, and other scientific users: to validate the mandatory EventsList and EventsTypes Objects; to validate the mandatory Attributes of each EventsList and EventsTypes Object; (following ISO standards) to validate common data standards to be used. Report at next CGMS (July 2013). ITERATIVE PROCESS

Thank You Any Questions?