Uncertainty in aerosol retrievals: interaction with the community Adam Povey 1, Thomas Holzer-Popp 2, Gareth Thomas 3, Don Grainger 1, Gerrit de Leeuw 4, and theAerosol_cci2 team 1 Atmospheric, Oceanic, and Planetary Physics, University of Oxford, Oxford 2 Earth Observation Center, DLR, Oberpfaffenhofen 3 RAL Space, Rutherford-Appleton Laboratory, Harwell 4 Climate Research, FMI, Helsinki

CMUG Integration Meeting, 27 th May 2015Summary of Aerosol_cci uncertainty workshop Status First ACCI2 (test) uncertainties validated — Valuable relative information — Absolute values too large — Need harmonization between different algorithms — Upgrades / harmonization in progress Critical discussion at AEROSAT - Two tendencies: - “try and error + improve” - Fall back on error envelopes

CMUG Integration Meeting, 27 th May 2015 Communication Users have varied, if compatible, needs. As the data volume increases, the desire for detailed uncertainty information decreases. — Uncertainty information is unimportant in decadal studies. Process studies can make detailed use of it. — Data assimilation requires a quantitative estimate of uncertainty. Some groups (ECMWF, NRL) prefer to do their own bias correction. — Most users desired a single value characterising the uncertainty on each quantity they studied.

CMUG Integration Meeting, 27 th May 2015 Communication Users have varied, if compatible, needs. As the data volume increases, the desire for detailed uncertainty information decreases. Though dissatisfying, it is not practically wrong to overestimate uncertainty. Producers felt quality flags were misunderstood. — Using only the “highest quality” data introduces bias as “quality” means “well-suited to analysis with this algorithm.” — Not all sources of error can currently be quantified.

AOD Uncertainties

CMUG Integration Meeting, 27 th May 2015 Sources of uncertainty Measurement — Random fluctuations in detector and electronics. Parameter — Uncertainties propagated from auxiliary data used by the retrieval (e.g. surface brightness). Approximation — Approximations in the forward model (e.g. LUTs). These are generally well-represented in analytic uncertainty estimates as they are easily quantified.

CMUG Integration Meeting, 27 th May 2015 Sources of uncertainty Resolution — Variations on scales smaller than that observed (pixel size, sampling). — Highly dependent on variable considered and the current state (natural variability can be relatively more important). — Most important for Level 3 products. Precisely how to propagate uncertainties into Level 3 is an open question for many products. Spatial and temporal correlations of errors mostly neglected (as poorly constrained). — Relevant to intercomparision of ground and satellite-based observations due to differing spatial/temporal coverage.

CMUG Integration Meeting, 27 th May 2015 Sources of uncertainty System — Impact of chosen aerosol microphysical properties. Not uncommon for a range of aerosol models to provide different AODs that each correspond to radiances consistent with observations. — Most satellite products require some cloud filtering. Cloud contamination will clearly introduce error. Radiative impacts known to extend well beyond cloud edges. Excessive cloud clearing will bias an averaged product against high radiance situations. Highly unclear how to provide any explicit estimation of these uncertainties.

Aerosol CCI Progress Meeting 4, 7 May 2015Lessons learnt on uncertainty L3 uncertainty ORAC: June 2008

Aerosol CCI Progress Meeting 4, 7 May 2015Lessons learnt on uncertainty L3 uncertainty ORAC: December 2008

CMUG Integration Meeting, 27 th May 2015 Outstanding issues Pixel level uncertainty sidesteps spatial/temporal correlations in error. — For example, if you average data over large time or spatial areas, does that increase or decrease net error? How can uncertainty at Level 3 be best characterized? Jacobian techniques assume errors are Gaussian. This is not true for some error terms. Current methods only address the ‘known unknowns’. How can the ‘unknown unknowns’ be addressed?

CMUG Integration Meeting, 27 th May 2015 Ensemble techniques Error propagation represents quantified errors. Illustrate impact of unquantified errors through multiple, self-consistent realisations of the data set. — Represents non-linear error propagation; — Allows user to apply their own knowledge to distinguish between different systems where otherwise unknown; — Users already familiar with ensemble techniques from model outputs.

CMUG Integration Meeting, 27 th May 2015 Ensemble techniques Important to consider retrieval (and model) averaging kernels when comparing data. — Otherwise, the quantities are not directly comparable and validation may confuse their differing definitions for bias Not all uncertainty information is quantitative. — Detailed quality flags, user guides important. e.g. “high AOD unreliable over desert”; unconverged retrieval To be submitted to AMT(D) in June.

CMUG Integration Meeting, 27 th May 2015 Looking forward There are significant uncertainties in the uncertainty of many retrievals. None the less, an estimate of uncertainty is required by users and for validation. Need to start with what can currently be achieved and iterate with users until we have usable uncertainty products. — Concentrate on how best to communicate where the algorithm is known to fail. Must improve reputation of remote sensing data among users. Providing validated pixel-level uncertainties and continuing the conversation about how to improve them should help.