The NOAA/NESDIS/STAR Long Term Strategy of Hyper Spectral Fundamental Climate Data Records and Environmental Climate Variables Antonia Gambacorta, Chris Barnet, Walter Wolf, Eric Maddy, Thomas King, Murty Divakarla, Mitch Goldberg NOAA/NESDIS/STAR Camp Springs, MD, USA 1

Outline  Part I. The NOAA/NESDIS/STAR long term hyper spectral strategy for climate applications  Part II. Temperature & water vapor retrieval products: characteristics and validation results 2

Part I The NOAA/NESDIS long term hyper spectral strategy for climate applications 3

2002 AIRS/AMSU retrieval algorithm into operations Migrate the AIRS/AMSU retrieval algorithm into operations for METOP IASI/AMSU/MHS phase A, B and C (2006, 2011, 2016) Metop A IASI operations approved on June 18 th 2008; operational since August 14 th 2008 Currently studying differences between instruments (AIRS, IASI and CrIS in simulation) Migrate the AIRS/IASI algorithm into operations for NPP ( October 25 th, 2011 ) & NPOESS (~2013,~2018) CrIS/ATMS/VIIRS (NOAA NDE program) NOAA/NESDIS/STAR long term hyper spectral sounding strategy AQUA 1:30pm orbit (May 4 th, 2002) METOP 9:30am orbit, (October 19 th 2006) NPP 1:30pm orbit (October 25 th, 2011)

The NOAA/NESDIS/STAR processing system is a modular design compatible with multiple instruments Retrieval products Diagnostic tools AIRS IASI CrIS Retrieval Code ECMWF NCEP SONDES ATOVS A long term uniform data record of atmospheric variables - cloud cleared radiances, temperature, water vapor, trace gases - by employing:  the same retrieval algorithm  the same underlying radiative transfer spectroscopy (UMBC SARTA)  a modular design for re-processing

6 Sequence of Steps of the Retrieval Algorithm  1) A microwave retrieval module which derives cloud liquid water flags and microwave surface emissivity uncertainty;  2) A fast eigenvector regression retrieval for temperature and moisture that is trained using the ECMWF analysis and observed cloudy radiances;  3) A cloud clearing module that uses a set of microwave and IR channels to produce the cloud-cleared IR radiance product and reject those cases violating the cloud-clearing requirements;  4) A fast eigenvector regression retrieval for temperature and moisture that is trained using the ECMWF analysis and IASI cloud cleared radiances;  5) The final IR retrieval module, which uses the regression retrieval as an initial solution and produces the final version of the physical retrieval by an iterated regularized least squared minimization.  We start with the temperature retrieval, because temperature is the most linear component of the RTA equation, followed by water vapor, ozone, etc.

7 Climate quality temperature and water vapor products require compelling instrument Specifications   Need High Spectral Resolution & Spectral Purity » »Ability to isolate spectral features → vertical resolution. » »Ability to minimize sensitivity to interference signals.   Need Excellent Instrument Noise & Instrument Stability » »Low NEΔT is required. » »Minimal systematic effects (scan angle polarization, day/night orbital effects, etc.)   Need Large Spectral Coverage (multiple bands) & High Sampling » »Increases the number of unique pieces of information. » »Ability to remove cloud and aerosol effects.   Need high accuracy in the retrieval of temperature, moisture and surface parameters » »Extracting small signals from noisy data » »Errors in the retrieval of temperature, water vapor and surface parameters are of the same order of magnitude of climate trend signal Dedicated efforts in selecting retrieval channel lists Dedicated efforts in validating temperature & moisture products

Instrument comparison (I): Information content of AIRS, IASI and CrIS AIRS, IASI and CrIS have comparable information content (first 500 eigenvalues shown for brevity. Global ensembles used for the SVD)  IASI highest information content  CrIS in simulation: reason for higher information content wrt AIRS

Instrument comparison (II): AIRS, IASI and CrIS instrument noise Comparable Longwave band noise – IASI shortwave band noise highest Channels used in retrieval scheme carefully selected to avoid high NEDT

NOAA/NESDIS channel selection methodology 10 Core water vapor lines are selected based on low NEDT and low interference from other atmospheric components. From spectroscopy, the interference signals of other species (methane, ozone, etc.) are well known and are used as off-diagonal terms of the noise covariance matrix this kind of error sensitivity analysis allows to discriminate correlation sources in within the retrieval scheme from natural signals. Retrieval error assessment : this kind of error sensitivity analysis allows to discriminate correlation sources in within the retrieval scheme from natural signals. Dashed vertical lines indicate selected water vapor channels Sensitivity to: Water vapor 10% perturbation 1k temperature perturbation 1K SST perturbation 10% ozone perturbation 2% methane perturbation

Instrument comparison (III): Water vapor channel selection used in the NOAA/NESDIS retrieval algorithm for AIRS, IASI and CrIS 11 AIRS version 5 IASI version 5 CrIS version 1 AIRS version 6 candidate water vapor channel list improvement Improvement work in progress Low water vapor sensitivity in the lower stratosphere (feature common to all latitudes)

12  Nominal Geometric Centroids obtained from the IASI Instrument Point Source Function (IPSF)  Radiometric Centroids obtained from the IIS measurement and the IASI IPSF  Spatial inhomogeneities introduce a shift between the geometric and radiometric centroids  The higher the spatial inhomogeneity, the largest the radiometric centroid shift IIS Imager (64x64 pixels) and IASI FOVs (black contour) Instrument Line Shape distortion in presence of FOV in-homogeneities: What’s error introduced in the radiance measurement?

13 ILS dependence on radiometric centroid shift  In general a non uniform light source introduces a distortion in the pixel ILS. The frequency shift of the peak is the dominant effect.  This frequency shift is a source of error in the radiance spectrum that we try to quantify (next slides). Radiometric centroid shift Geometric centroid angular position simplified case of an ideal monochromatic source

IASI Instrument Line Shift Effects in presence of non uniform scenes Band 1 Band 2 Band 3 MEAN = rad SIGMA = 0.1 mrad Error becomes noticeable at 3 sigma shift FOV Radiometric Centroid shift Error becomes noticeable at 3 sigma shift Error becomes noticeable at 2 sigma shift NEDT Error induced is negligible except for very rare cases (that can be flagged out)

Part II The NOAA/NESDIS Temperature and Water Vapor retrieval products: characteristics and validation results 15

16 NOAA/NESDIS water vapor and temperature key features for climate applications  Top of atmosphere to surface vertical retrievals of temperature and water vapor »The full vertical extent of the retrieval profiles allows to perform a complete study of water vapor’s sensitivity to temperature variations in the upper, middle and lower troposphere.  Cloud-clearing algorithm allows for spatial uniformity »no clear-sky bias typical of IR hyper spectral sensors.  Vertical resolution »High vertical resolution is fundamental to capture the vertical structure in the climate trends of temperature and water vapor »AIRS v5 tropospheric temperature (moisture) retrieval resolution, as determined by the full- width-half-maximum of the averaging kernels, ranges between ~2.5km (3km) near the surface and 6km (4km) near the tropopause (Ref.: Maddy and Barnet, 2007)  High retrieval accuracy »Extensively validated retrieval algorithm (see ahead) (Ref: Fetzer et al., 2003; Divakarla et al., 2006; Tobin et al., 2006; Fetzer et al., 2008; Gambacorta et al., 2008, etc.) A robust temperature and water vapor data set A robust temperature and water vapor data set for climate applications for climate applications

IASI Temperature and Water Vapor vs ECMWF 17 Global Ensemble – Ocean Night Only – Ocean Night Clear Only Results consistently improve for both T and q due to better characterized surface emissivity and no cloud clearing errors; physical retrieval improves from first guess Solid: physical retrieval; Dashed: First guess TemperatureWater Vapor RMS statistics

IASI Temperature and Water Vapor vs ECMWF 18 Global Ensemble – Ocean Night Only – Ocean Night Clear Only Results consistently improve for both T and q due to better characterized surface emissivity and no cloud clearing errors; physical retrieval improves from first guess Solid: physical retrieval; Dashed: First guess TemperatureWater Vapor SDV statistics

IASI Temperature and Water Vapor vs ECMWF 19 Global Ensemble – Ocean Night Only – Ocean Night Clear Only Ocean night Clear only water vapor mid-trop bias degradation due to low statistics (case by case study not shown) Solid: physical retrieval; Dashed: First guess TemperatureWater Vapor BIAS statistics

IASI validation vs ECMWF by latitudinal regions 20 TemperatureWater Vapor Polar region – Mid latitude region – Tropics Solid: physical retrieval; Dashed: Night Ocean Only RMS statistics Results consistently improve from high to low latitudes:  Surface emissivity better characterized; sharper vertical temperature gradient; less uniform cloud formations

IASI validation vs ECMWF by latitudinal regions 21 TemperatureWater Vapor Polar region – Mid latitude region – Tropics Solid: physical retrieval; Dashed: Night Ocean Only SDV statistics Results consistently improve from high to low latitudes:  Surface emissivity better characterized; sharper vertical temperature gradient; less uniform cloud formations

IASI validation vs ECMWF by latitudinal regions 22 TemperatureWater Vapor Polar region – Mid latitude region – Tropics Solid: physical retrieval; Dashed: Night Ocean Only BIAS statistics Results consistently improve from high to low latitudes:  Surface emissivity better characterized; sharper vertical temperature gradient; less uniform cloud formations

23 RAOBs Validation courtesy of Murty Divakarla  Bias and RMS difference Statistics with Reference to RAOBs. »Statistics for MetOp-IASI (9:30AM/PM) –Acceptance Criteria: Mid-Troposphere Temp Flag = 0 –Solid Lines for IASI (9:30 AM/PM) l RAOB vs. IASI Final - Physical Retrievals l RAOB vs. IASI Fast Regression (FG) l RAOB vs. MetOp-ATOVS Retrievals l RAOB vs. ECMWF Forecast l RAOB vs. NCEP-GFS Analysis/Forecast Fields »Similar Statistics for Aqua-AIRS (1:30 PM/AM) –Dotted Lines l Similar Color Convention –Acceptance Criteria: Mid-Troposphere Temp Flag = 0 –(I do have comparison with V5 QA also) »These Figures are only a few selected ones. Go to our website to see RMS and Bias Plots for –Any region (tropics, midlat, polar, global) –Any category: land/sea; day/night etc.

24 IASI- T(p), q(p) RMS Difference Global (L+S+Coast) NSAMP=12,666 Yield: 55% Acceptance Criteria: Mid-Troposphere Temp Flag = 0 RAOB vs. IASI AVN ATOVS ECMWF FG

25 AIRS- T(p), q(p) RMS Difference Global (L+S+Coast) NSAMP=5,993, Yield: 36% Acceptance Criteria: Mid-Troposphere Temp Flag = 0 RAOB vs. AIRS AVN ATOVS ECMWF FG Dotted Lines : AIRS

26 IASI- T(p), q(p) RMS Difference Sea Samples NSAMP=2,848 RAOB vs. IASI AVN ATOVS ECMWF FG

27 AIRS - T(p), q(p) RMS Difference Sea Samples NSAMP=540 RAOB vs. AIRS AVN ATOVS ECMWF FG

IASI Validation Summary 28  Globally (except high latitude regions) »Temperature rms: ~ 1.5 K (lower troposphere) to ~1k (upper troposphere) »Temperature bias: <+/-0.5 »Water vapor rms: <10% (lower troposphere) to 25% (upper troposphere) (vs ECMWF) »Water vapor bias: < 10% (lower troposphere) to +/- 3% (upper troposphere) (vs ECMWF)  AIRS shows comparable results, 10% degraded in the mid-upper tropospheric water vapor profile (improvements are underway).  Ultimate target: meet the GCOS climate variables specification requirements globally

Conlcusions  NOAA/NESDIS temperature and water vapor characteristics:  Top of atmosphere to surface vertical retrievals of temperature and water vapor  Cloud-clearing algorithm »allows for spatial uniformity no clear-sky bias typical of other IR hyper spectral data bases.  Error Sensitivity analysis »allows to discriminate between correlation sources in within the retrieval scheme from natural signals.  Vertical resolution »High vertical resolution is fundamental to capture the vertical structure in the dynamics and climate trends of temperature and water vapor  Re-processing capability »Allows to keep data sets updated with latest optimization of the retrieval algorithm  High retrieval accuracy »The NOAA/NESDIS temperature and water vapor products show promising results towards fully meeting the GCOS climate variables specification requirements almost globally. 29

30