evaluation aerosol CCI retrievals Reading 2012

the participants / the task AATSR F v142 AATSR O v202/v2q2 AATSR S v040/v031 MERIS A v21 MERIS B v11 MERIS E 802 PARASOL v30 MODIS_aqua c5.1 MODIS_terra c5.1 SEAWIFS MISR v31 evaluate daily global ‘CCI-aerosol’ retrievals –for the entire year 2008 AOD 550nm Angstrom (if offered) versus AERONET / MAN performance in context of established retrievals

ECMWF AOD FBOV (single assim.)FMNG (dual assim.) less aerosol

ECMWF Angstrom FBOV (single assim.)FMNG (dual assim.) larger sizes

data content AOD year 2008 MISR MODIS TERRA PARASOL MERIS-ESA MERIS-BAER MERIS-ALAMO ATSR-SU ATSR ORAC ATSR ADV

data volume AOD 1/1/2008 MODIS AQUA MISR PARASOL MERIS-ESA MERIS-BAER MERIS-ALAMO ATSR-SU ATSR ORAC ATSR ADV MODIS TERRA

the scoring concept establish 1x1 gridded AERONET daily data –default: use +/- 30min of overpass of satellite –option: use daily averages identify local data pairs to level 3 daily 1x1 satellite retrievals (test data & reference data) require at least 10 events to perform a statistical analysis (e.g. correlation, bias)

the scoring concept establish 1x1 gridded AERONET daily data –default: use +/- 30min of overpass of satellite –option: use daily averages identify local data pairs to level 3 daily 1x1 satellite retrievals (test data & reference data) require at least 10 events to perform a statistical analysis (e.g. correlation, bias)

scoring steps establish regional scores (1: best, 0: poor) –Step1regional spatial score for each day (rank) correlation and (rank) bias –Step2regional temporal score for each 1x1 (rank) correlation and (rank) biast –Step3combined regional score total score = bias * spat.corr * temp.corr –Step4combine regional to one global score ‘single score’ from regional total scores investigated property: AOD550

MODIS TERRA AOD performance total error –bias –spatial –temporal bias –tendency –albsolute

AOD bias versus AERONET smaller   larger MODIS TERRA MISR PARASOL MERIS-ESA MERIS-BAER MERIS-ALAMO ATSR-SU ATSR-ADV ATSR-ORAC

AOD difference to MODIS smaller than MODIS larger than MODIS PARASOL ATSR-SU ATSR-ORAC ATSR-ADV

AOD total error comparison  Increasingly larger ERROR MODIS TERRA MISR PARASOL MERIS-ESA MERIS-BAER MERIS-ALAMO ATSR-SU ATSR-ORAC ATSR-ADV

overall AOD scores global land ocean MODIS A/T AATSR S v AATSR O v MISR AATSR F v AATSR S v MERIS B v SEAWIFS PARASOL MERIS ESA MERIS A based on a 10+ sample statistics absolute larger score is better +/- sign for overall bias

let us go regional (1) boreal – MODIS (.47), MERIS-BAER(.22) N.America temp –ATSR-S40 (-.62), ATSR-O202 (.58), ATSR-F142 (-.57), MISR (.56), MODIS (.44) S.America trop –Modis (-.67) S.America temp –MODIS (.44), Seawifs (.34), MERIS-BEAR (.31) N.Africa –ARSR-S31 (-.72), ATSR-S40(.70), MOD (.67), BEAR (.37) S.Africa –MODIS (.63) 10 + events

let us go regional (2) EU-Asia boreal – MODIS (-.58) EU-Asia temp –MODIS (.63), MISR (-.62), ATSR-S40 (-.58), ATSR-F142 (-.54), ATSR-O202 (-.53), ATSR-S31 (-.52), BAER (-.41) Asia trop –Modis (-.62), MERIS-BEAR (.47) Europe –MISR (-.63), ATSR-S40 (-.61), ATSR-F142 (-.59), ATSR- O202 (.59), MODIS (.58), ATSR-S31 (-.53), BEAR (.31) N.Pacific temp –MODIS (.61), ARSR-F142 (-.54), MISR (.53), BEAR (.47) N.Atlantic temperate –MODIS (.61), SEAWIFS (.47), BEAR (.45) 10 + events

what regional scores tell us ATSR data coverage is poor compared to MODIS comparisons in regions with sufficient data statistics indicated that all three ATSR AOD data-sets in quality are at/near the level of MODIS/MISR current overall ATSR AOD ranking: –1. SU4.0 –2. O202 –3. F142 MERIS potentially offers data similar to MODIS MERIS BEAR 1.1 demonstrates data-volume processing … but data-quality is relatively poor

overall Angstrom scores global land ocean AATSR S v AATSR S v AATSR F v AATSR O v MISR MERIS ESA PARASOL MERIS A all Angstrom data are biased HIGH! land scores are poorer than ocean scores based on a 10+ sample statistics absolute larger score is better +/- sign for overall bias

final thoughts some of the current CCI retrievals for AOD have reached the maturity of US products while ATSR has become competitive, MERIS products lag behind ocean reference data are needed (MAN data are too sparse) to statistically evaluate retrieval quality over oceans, also in order to get reliable scores for PARASOL and MERIS data

extra slides

further thoughts the combination of ‘sufficient’ coverage and of ‘sufficient’ accuracy matters temporal correlation (seasonality) is generally poorer than spatial correlation (patterns)

issues more solid (statistical) evaluations require … better spatial coverage –how to get references for non AERONET loc ? better temporal coverage –4 months are not enough for limited swath better accuracy –more confidence in AERONET reference –fewer sites, less variability, several per hour

2ass … minus … 1 ass  improvement deteriation 

2 ass … minus … 1 ass  improvement deteriation 

O202 … minus … SU40  improvement deteriation 

O202 … minus … SU40  improvement deteriation 

O202 … minus … SU40  improvement deteriation 

SU40 … minus … SU31  improvement deteriation 

F142 … minus … SU40  improvement deteriation 

extras this Thn is this Thn is Thn is Thn is

MODIS AQUA AOD performance total error –bias –spatial –temporal bias –tendency –absolute

MISR AOD performance total error –bias –spatial –temporal bias –tendency –absolute

POLDER AOD performance total error –bias –spatial –temporal bias –tendency –absolute

MERIS ESA AOD performance total error –bias –spatial –temporal bias –tendency –absolute

MERIS BEAR AOD performance total error –bias –spatial –temporal bias –tendency –absolute

MERIS ALAMO AOD performance total error –bias –spatial –temporal bias –tendency –absolute

AATSR SU40 AOD performance total error –bias –spatial –temporal bias –tendency –absolute

AATSR SU31 AOD performance total error –bias –spatial –temporal bias –tendency –absolute

AATSR O202 AOD performance total error –bias –spatial –temporal bias –tendency –absolute

AATSR O202iq AOD performance total error –bias –spatial –temporal bias –tendency –absolute effectively ‘identical to O202

AATSR ADV142 AOD performance total error –bias –spatial –temporal bias –tendency –absolute

AATSR SU40 AOD performance total error –bias –spatial –temporal bias –tendency –absolute MODIS terra

AATSR O202 AOD performance total error –bias –spatial –temporal bias –tendency –absolute MODIS terra

AATSR ADV142 AOD performance total error –bias –spatial –temporal bias –tendency –absolute MODIS terra

POLDER AOD performance total error –bias –spatial –temporal bias –tendency –absolute MODIS terra