AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M. Divakarla

Science Activities  Data compression.  Validate and improve radiative transfer calculations.  Cloud detection and clearing.  Cloud products  Channel selection (super channels).  Validate and improve retrieval algorithms.  Trace gases  Surface emissivity  Use MODIS to improve AIRS cloud detection and cloud clearing  Radiance bias adjustments  Forecast impact studies

TOPICS  Use of principal components (a.k.a. eigenvectors) for data compression.  Surface emissivity  Cloud detection

AIRS Geophysical Products  Microwave-only retrieval of sfc emissivity, sfc temperature, sfc type and profiles of temperature, water vapor and cloud liquid water.  AIRS retrieval of cloud amount and height, sfc emissivity, sfc temperature, and profiles of temperature, water vapor and ozone.  AIRS has two retrieval steps – very fast eigenvector regression followed by a physical retrieval algorithm.

Data Compression  Advanced IR sounder data are very large compared with current sounders (1 orbit ~ 2GB vs. 8 MB) Much larger for GIFTS.  Information is not independent. Principal component analysis (PCA) is often used to reduce data vectors with many components to a different set of data vectors with much fewer components that still retains most of the variability and information of the original data  Data are rotated onto a new set of axes, such that the first few axes have the most explained variance.  Principal component scores are provided instead of the individual channels.  Individual channels can be reconstructed with minimal signal loss with added benefit of noise reduction.

Generating AIRS eigenvectors  Collect an ensemble of AIRS spectra (2378 channels).  The radiances are normalized by expected instrumental noise (signal to noise)  Compute the covariance matrix S  Compute the eigenvectors E and eigenvalues  S = E  E T  E = matrix of orthonormal eigenvectors (2378x2378)  = vector of eigenvalues (explained variance)

Training Ensemble  Eigenvectors are generated from a spatial subset of AIRS data (200 mbytes vs 30 GB full data)  Eigenvectors are generated daily.  A static set of eigenvectors is used, but the ensemble is occasionally updated with new structures.  When the ensemble is updated a new set of eigenvectors is also updated.

Locations used in generating eigenvectors

Applying AIRS eigenvectors  On independent data – compute principal component scores.  P = E T R ; elements of R = (r i - r i ) /n i  Invert equation and compute reconstructed radiances R*.  R* = E P  Reconstructed radiances are used for quality control.  Reconstruction score = [ 1/N 3(R* i - R i ) 2 ] 1/2 i = 1 ….N channels

Square root of the eigenvalues

 Reconstruction score = [ 1/N 3(R* i - R i ) 2 ] 1/2 i = 1 ….N channels

 Reconstruction score = [ 1/N 3(R* i - R i ) 2 ] 1/2 i = 1 ….N channels

Monitoring Eigenvectors  Monitoring eigenvectors is critical  Eigenvectors may need to be updated due to new structures that were not in the original ensemble

12/4/00 reconstruction scores

Monitoring reconstruction score is important Days July Aug Sep Oct Nov Dec Jan Feb

Noise Noise free 75 PCS

Observed vs noise-free reconstructed vs noise-free. Noise Reduction

“ Observed” Reconstructed

Observed vs. Reconstructed

New Plan  Generate full spatial resolution AIRS principal component score datasets  Size ~ 5 MB instead of 150 MB per six minute granule

Surface emissivity

Retrieval error based on 18 channels Background Std dev. Retrieval error

Clear detection

BACKGROUND  NWP centers will assimilate clear radiances  Need very good cloud detection algorithm  Very important for radiance validation and to initiate the testing of the level 2 retrieval code.

Cloud Detection over Ocean  Use VIS/NIR channels during day.  Compare SST with 2616 cm-1 at Night.  Predicting SST from 11 and 8 micron channels (works for day and night)  Predict 2616 from 8 micron channels (night)  11 micron window > 270 K

ONLY 0.5% residual clouds

Cloud detection – Non Sea  Predict AIRS channel at cm-1 from AMSU  FOV is labeled “mostly clear” if predicted AIRS – observed AIRS < 2 AND IF  SW LW IR window test is successful: [ch( )-CH( )] < 10 K  Variability of radiance within 3x3 <

Clear Detected Fovs Cloud cleared cases

Future Work – Merge MODIS and AIRS  High spatial resolution will improve determination of clear AIRS fovs.  High spatial resolution will greatly improve clear estimate needed for cloud clearing.

MODIS Sounder Radiance Product  MODIS has HIRS-like sounder channels – but at high spatial resolution (1 km).  Find a few clear MODIS fovs in a 50 x 50 km area should provide a yield of 80% -- similar to AMSU

Summary  Busy getting ready for real AIRS data  Simulating AIRS in real-time has provided a means to develop, test and validate the delivery of products to NWP centers,  AND created a platform to develop scientific tools to analyze the data and test algorithms.  Early releases of the data should be available 3 months after launch  Final radiance products ~ 7 months  Retrievals ~ 12 months  First activity will be to examine biases between measured and computed radiances and validation of the clear detection algorithm.  “Day-2” Utilize MODIS