Defining the Products: ‘GSICS Correction’ Tim Hewison Marianne König

Scope What we need to decide: Draft Action GDWG03_17: GRWG to define the content of their NetCDF files with support from the GDWG. End of March 2009. Includes input and output datasets and intermediate files

Generic Data Flow GRWG to define content NetCDF files Includes input and output datasets and intermediate files Related to Definition of the ‘Product’ The ‘GSICS Correction’ i.e. Last Step in Data Flow Other Products? (plots, tables, etc) What needs to be published? Correction algorithm Input parameters needed (‘Statistical Data’) Zoom on Last Step Ensure we are clear Can we make in generic?

Generic Data Flow Applying the Correction Statistical Data User Input From time-series of inter-calibrations: Satellite/Instrument/Ch Time Series of Regression Coefficients Version numbers Users take the GSICS Correction and apply it to input data: Generic Data Flow User Input Satellite/Instrument/Ch Date/Time Geometry (Lat/Lon/IncAng/AzAng/PolAng/SunAng, …) Radiances/Counts Correction Algorithm e.g. Look-Up Table, FORTRAN subroutine, New calibration coefficients, … Is this generic? What is missing? How can Correction be Implemented? Distributed? Corrected Radiances With Uncertainties

Defining ‘Statistical Data’ If we are happy with this generic form of correction Need to Define ‘Statistical Data’: Contents Satellite/Instrument/Channel Time Series of Regression Coefficients with Uncertainties Version numbers Anything else? Format NetCDF To be defined by GDWG

How Could Correction be applied? In general, the correction is a function of multiple variables e.g. Date, Time, Radiance, Incidence Angle, etc… Need to establish sensitivity of bias to each of these Need to develop algorithms to apply GSICS Corrections Although these algorithms do not all need to be the same, we can consider generic correction methods: Divide dataset into homogeneous chunks and correct each independently E.g. Monthly mean biases, near-nadir, over sea, in tropics, etc… Combine all inter-calibration results and develop multivariate correction E.g. Multiple Linear Regression, Kalman Filter, etc. Some combination of these E.g. If sensitivities & non-linearity constant with time

Generic Trend Correction Algorithm New Result Add to Time Series Trend Significant Trend Insignificant Bias Significant Time-dependent correction Constant correction Bias Insignificant Time-dependent correction? No correction Inconsistent Consistent Test Consistency with Previous Results Calc Trend in Results Sound Alarm Reset Trend Stats Calc Timescale over which trend produces significant changes Insignificant Test significance of Bias Smooth results over this timescale Could apply same method to correct other dependences in the calibration e.g. Time of Day, Latitude, Scan angle,… No Correction Significant Test significance of Trend Time-dep. Correction Constant Correction Significant Insignificant

General Structure of ‘Statistical Data’ What do we need in ‘Statistical Data’ to apply a correction? In general, the correction may be a function with multiple inputs: Satellite, Instrument, Channel Date, Time Geometry (Lat ,Lon ,IncAng, AzAng, PolAng, SunAng, …) Radiances/Counts Need to establish which of these input parameters are inter-dependent Can we assume geometric corrections are constant with time, radiance? Can we assume non-linearities are constant with time, geometry? Difficult to automate this decision process May be different for application to Near Real-time and Archive data

Proposed Content of ‘Statistical Data’ What do we need in ‘Statistical Data’ to apply a correction? Time Series of Regression Coefficients (with uncertainties) (Can build Look Up Table from this??) Statistics of these time series: Date/time of last trend reset Alert flag for results not consistent with trend Time constant over which inter-calibrations should be averaged? Trend statistics – slope, offset (with uncertainties) Sensitivity of bias to radiance (with uncertainties) Sensitivities of bias to geometric factors (with uncertainties) Lat, Lon, IncAng, AzAng, PolAng, SunAng, … Assume these sensitivities to be constant within time constant (??) What does correction mean in terms of Tb bias for Reference Scenes? (±)

Draft NetCDF format for Inter-Calibration Results Coalesce time series of instantaneous results into ‘Statistical Data’ file and include: Trends & Statistics of time series Sensitivity of bias to other factors Calculated over longer periods

Discussion Points Is this concept generic? Could it be applied to your inter-calibration to produce a GSICS Correction? What ‘Statistical Data’ is needed to apply GSICS Corrections? Time Series of Regression Coefficients Trends and Statistics of these time series Sensitivity of bias to radiance, geometry, etc. Can we define sensitivity of results in a generic way? To factors such as time, geometry and radiance These variables can be inputs into the Correction functions How can Correction be Implemented? E.g. Start simple and aim to evolve? Distributed?