MPI-Meteorology Hamburg, Germany Evaluation of year 2004 monthly GlobAER aerosol products Stefan Kinne

the task an evaluation of  GlobAER 2004 global maps for aerosol optical depth(info on amount) for Angstrom parameter(info on size)  by ASTR (dual view, global, at best 10 per month)  by MERIS (nadir view, global, at best 1 per day)  by SEVIRI (nadir view, regional, at best 30 per day)  by a merged (ATSR / MERIS / SEVIRI) composite

the questions  how well do the data compare to trusted data references (e.g. AERONET) ?  how well do the data compare to existing data sets – even for same sensor data ?  can the performance be quantified ? more specifically …  what are the scores of a new (outlier- resistant) method … examining bias, spatial and temporal variability ?

the investigated properties  aerosol optical depth (AOD) extinction along a (vertical) direction due to scattering and absorption by aerosol  here for the entire atmosphere  here for the mid-visible (0.55  m wavelength)  Angstrom parameter (Ang) spectral dependence of AOD in the visible spectrum  small dependence (Ang ~ 0)  aerosol > 1  m size  strong decrease (Ang > 1.2)  aerosol < 0.5  m size

monthly data-sets GlobAER 2004 maps  GAa - ATSR  GAs - SEVIRI  GAm - MERIS  GAx – merged other multi-ann. maps  med – model median  clim – med & aer(sun)  sky – med & aer(sky)  TO – TOMS other 2004 maps  ATs - ATSR Swansea  SEb- SEVIRI Bruxelles  Mdb - MODIS deep blu  MO - MODIS std coll.5  MI – MISR version 22  Ag – AVHRR, GACP  Ap – AVHRR, Patmos  Aer – AERONET 2004

AOD map comparisons  AOD annual maps all available data  AOD seasonal maps ATSR GlobAER vs Swansey SEVIRI GloabAER vs RUIB-Bruexelles  difference maps … to a remote sensing ‘best’ composite all available data-sets focus on the four GloabAER products

AOD – 2004 annual maps

ATSR / SEVIRI – seasonal AOD

AOD diff to ‘composite’  underestimates overestimates 

quick (annual) AOD check  ATSR underestimates in dust regions overestimate in biomass regions  SEVIRI severe biomass overestimates useful over-land estimates ?  MERIS apparent land snow cover issue  merged not the envisioned improvement  ‘-’ ’+’ 

Angstrom map comparisons  Angstrom annual maps all available data  Angstrom seasonal maps ATSR GlobAER vs Swansey SEVIRI GloabAER vs RUIB-Bruexelles  difference maps … to a climatology (model & AERONET) all available data-sets focus on the four GloabAER products

Angstrom – 2004 annual maps

ATSR / SEVIRI – seasonal Angstr.

Angstrom diff to ‘climatology’  underestimates overestimates ....

quick annual Angstrom check  ATSR underestimates in tropics overstimates in south. oceans  SEVIRI underestimates over oceans strong overestimates over land  MERIS overestimates over land  merged not the envisioned improvement  ‘-’ ’+’ 

the SCORING challenge  quantify data performance by one number  develop a score such that contributing errors to be traceable back to bias spatial correlation temporal correlation spatial sub-scale (e.g. region) temporal sub-scale (e.g. month, day)  make this score outlier resistant

one number !

info on overall bias sign of the bias

| 1 | is perfect …. 0 is poor sign of the bias the closer to absolute 1.0 … the better

product of sub-scores = 0.9 *- 0.7 * 0.8 the closer to absolute 1.0 … the better temporal correlation sub-score bias sub- score spatial correlation sub-score sign of the bias

spatial stratification = 0.9 * -0.7 * 0.8 time score bias score spatial score spatial sub-scale scores overall score regional surface area weights TRANSCOM regions

temporal stratification = 0.9 * -0.7 * 0.8 time score bias score spatial score spatial sub-scale scores overall score temporal sub-scale scores (e.g. month or days) averaging in time instantaneous median data

sub-score definition  each sub-score S is defined by an error e and by an error weight w 0.9 * -0.7 * 0.8 S = 1 – w * e time score S bias score S spatial score S spatial sub-scale scores temporal sub-scale scores (e.g. month or days) instantaneous median data

definition of errors e  S = 1 – w * e   all values for the errors e are rank - based for “time score” and “spatial score” rank correlation coefficients for data pairs are determined  e, correlation = (1- rank_correlation coeff.) /2 (correlated: e = 0, anti-correlated: e = 1) time score bias score spatial score

definition of errors e  S = 1 – w * e for “bias score” all all data-pairs are placed in a single array and ranked by value then ranks are separated according to data origin, summed and (rank-sums are) compared  e, bias = (sum1 – sum2) / (sum1 + sum2) (strong neg.bias e = -1, strong pos.bias e= +1) an example (“how does the rank bias error work ?”)  set 1: value: rank-sum 1: 11  set 2: rank: rank-sum 2: 10 e = (1-2)/(1+2) = (11-10)/21 ~zero  no clear bias time score bias score spatial score

definition of error weight w  S = 1 – w * e  w is a weight factor based on the inter-quartile range / median ratio w = (75%pdf - 25%pdf) / 50%pdf … but not larger than 1.0 (w<1.0) simply put … if there is no variability an error does not matter bias score spatial score time score

scoring summary  one single score …  … without sacrificing spatial and temporal detail !  stratification into error contribution from bias spatial correlation temporal correlation  robustness against outliers still … just one of many possible approaches  now to some applications …

questions  how did GlobAER products score? overall ? seasonality ? spatial correlation ? bias ? in what regions ? in what months ?  how did scores place to other retrievals … with the same sensor (for the same year 2004) with other sensors (for the same year 2004)

selective evaluations for year 2004 data ..atsrSwansey ATSR ..gaatGlobAER ATSR 3/2009 (std) ..gaa2GlobAER ATSR 7/2009 (filtered) ..gaa3GlobAER ATSR8/2009 (test) ..misrMISR ver.22  how does ‘gaa2’ score ?  score diff between ‘gaa2’ and ‘misr’  score diff between ‘gaa2’ and ‘gaat’  score diff between ‘gaa2’ and ‘atsr’

performance vs AERONET GlobAER ATSR … “gaat” AOD score:.47 ANG score: -.57 Swansey ATSR … “atsr” AOD score:.57 ANG score: -.49 GlobAER ATSR (filter) … “gaa2” AOD score: none * ANG score: none * MISR v22 AOD score:.59 ANG score:.67 * too few areas with scores

side by side let us compare AOD details …  what is the sign of the bias ?  what is the bias strength (median diff) ?  what is the bias error ?  what is the spatial variability error ?  what is the seasonality error ?  what is the overall error  what are selected error differences among different retrieval products?

AOD bias sign vs AERONET ATSR GlobAER ATSR Swansey MISR vers.22 overestimateunderestimate ATSR GlobAER filtered

AOD bias error vs AERONET ATSR GlobAER ATSR Swansey MODIS coll.5 MISR vers.22

AOD bias strength vs AERONET ATSR GlobAER ATSR Swansey MISR vers.22

AOD bias error vs AERONET ATSR GlobAER ATSR Swansey MISR vers.22 ATSR GlobAER filtered

AOD spatial var. Vs. AERONET ATSR GlobAER ATSR Swansey MISR vers.22 ATSR GlobAER, filtered

AOD season error vs AERONET ATSR GlobAER ATSR Swansey MISR vers.22

AOD total error vs AERONET ATSR Swansey MISR vers.22 ATSR GlobAER filtered ATSR GlobAER

AOD ATSR GlobAER filtered - total error

AOD ATSR GlobAER filtered - bias error

AOD ATSR GlobAER filtered - spatial error

AOD summary  ATSR products ATSR is most promising (merged is poorer!)  ATSR AOD of GLOBAER poorer than the ATSR AOD by Swansey much poorer than MODIS or MISR filtered version scores poorer mainly due to deterioation in spatial and temp. variability  ATSR AOD by GlobAER with filter bias error is reasonable spatial variability is poor seasonality is poor

extras  score differences … for overall errors neg. difference  smaller error pos. difference  larger error  gaa2 vs atsr  gaat vs atsr  gaa2 vs gaat  gaat vs misr

gaa2 (filtered) vs atsr (swansey) black  better worse  green

gaat (globaer) vs atsr (swansey) black  better worse  green

gaa2 (filtered) vs. gaat (globaer) bb black  better worse  green

gaat (globaer) vs. misr  better worse 

AODATSR – GlobAER 2004

AODSEVIRI – GlobAER 2004

AODMERIS – GlobAER 2004

AODmerged – GlobAER 2004

AODSEVIRI – RUIB 2004

AODATSR – Swansea 2004

annual maps – AOD 2004 med median cli ‘climatology’ sunsun-photo Globaer products MOMODIS Efcforecast MIMISREasassimilation SEVIRIAATSRMERIS

annual maps – Angstrom 2004 med median cli ‘climatology’ sunsun-photo Globaer products MOMODIS Efcforecast MIMISREasassimilation AATSRMERISSEVIRI