1. Radar/lidar/radiometer retrievals of ice clouds from the A-train
Robin Hogan, Julien Delanoe
Department of Meteorology, University of Reading, UK
Richard Forbes
European Centre for Medium Range Weather Forecasts
Alejandro Bodas-Salcedo
Met Office, UK

2. Motivation
Clouds are important for climate due to interaction with radiation
A good cloud retrieval must be consistent with broadband fluxes at surface and top-of-atmosphere (TOA)
Advantages of combining radar, lidar and radiometers
Radar ZD6, lidar b’D2 so the combination provides particle size
Radiances ensure that the retrieved profiles can be used for radiative transfer studies
How do we do we combine them optimally?
Use a “variational” framework: takes full account of observational errors
Straightforward to add extra constraints and extra instruments
Allows seamless retrieval between regions of different instrument sensitivity
In this talk a new variational radar-lidar-radiometer algorithm is applied to a month of A-Train data
Comparison with MODIS retrievals
Evaluation of Met Office and ECMWF model ice clouds
Investigation of the morphology of tropical cirrus

3. Surface/satellite observing systems
Ground-based sites
ARM and Cloudnet
NASA A-Train
Aqua, CloudSat, CALIPSO, PARASOL
ESA EarthCARE
For launch in 2013
Radar
35 and/or 94 GHz
Doppler, Polarization
94 GHz CloudSat
94 GHz CPR
Doppler
Lidar
Usually
532 or 905 nm
Polarization
532 & 1064 nm CALIOP
355 nm ATLID
Polarization, HSRL
VIS/IR radiometers
Some have infrared radiometer, sky imager, spectrometer
MODIS, AIRS,
CALIPSO IIR (Imaging Infrared Radiometer)
Multi-Spectral Imager (MSI)
Microwave radiometers
Dual-wavelength radiometer
(e.g. 22 & 28 GHz)
AMSR-E (6, 10, 18, 23, 36, 89 GHz)
None
Broadband radiometers
Surface BBR
Europe/Africa sites have GERB overhead
CERES (TOA only)
BBR (TOA only)
Broadband radiometers used only to test retrievals made using the other instruments

4. Why not invert the lidar separately?
“Standard method”: assume a value for the extinction-to-backscatter ratio, S, and use a gate-by-gate correction
Problem: for optical depth d>2 is excessively sensitive to choice of S
Exactly the same instability for radar (Hitschfeld & Bordan 1954)
Better method (e.g. Donovan et al. 2000): retrieve the S that is most consistent with the radar and other constraints
For example, when combined with radar, it should produce a profile of particle size or number concentration that varies least with range
Implied optical depth is infinite

5. Formulation of variational scheme
For each ray of data we define:
Observation vector • State vector
Elements may be missing
Logarithms prevent unphysical negative values
Attenuated lidar backscatter profile
Ice visible extinction coefficient profile
Ice normalized number conc. profile
Extinction/backscatter ratio for ice
Radar reflectivity factor profile (on different grid)
Visible optical depth
Infrared radiance
Radiance difference
(TBD) Aerosol visible extinction coefficient profile
(TBD) Liquid water path and number conc. for each liquid layer

6. The cost function + Smoothness constraints
The essence of the method is to find the state vector x that minimizes a cost function:
Each observation yi is weighted by the inverse of its error variance
The forward model H(x) predicts the observations from the state vector x
Some elements of x are constrained by an a priori estimate
This term penalizes curvature in the extinction profile
+ Smoothness constraints

7. xk+1= xk+A-1{HTR-1[y-H(xk)]
Solution method
New ray of data
Locate cloud with radar & lidar
Define elements of x
First guess of x
An iterative method is required to minimize the cost function
Forward model
Predict measurements y from state vector x using forward model H(x)
Predict the Jacobian H=yi/xj
Gauss-Newton iteration step
Predict new state vector:
xk+1= xk+A-1{HTR-1[y-H(xk)]
-B-1(xk-b)-Txk}
where the Hessian is
A=HTR-1H+B-1+T
Has solution converged?
2 convergence test No
Yes
Calculate error in retrieval
Proceed to next ray

8. Radar forward model and a priori
Create lookup tables
Gamma size distributions
Choose mass-area-size relationships
Mie theory for 94-GHz reflectivity
Define normalized number concentration parameter
“The N0 that an exponential distribution would have with same IWC and D0 as actual distribution”
Forward model predicts Z from extinction and N0
Effective radius from lookup table
N0 has strong T dependence
Use Field et al. power-law as a-priori
When no lidar signal, retrieval relaxes to one based on Z and T (Liu and Illingworth 2000, Hogan et al. 2006)
Field et al. (2005)

9. Lidar forward model: multiple scattering
90-m footprint of Calipso means that multiple scattering is a problem
Eloranta’s (1998) model
O (N m/m !) efficient for N points in profile and m-order scattering
Too expensive to take to more than 3rd or 4th order in retrieval (not enough)
New method: treats third and higher orders together
O (N 2) efficient
As accurate as Eloranta when taken to ~6th order
3-4 orders of magnitude faster for N =50 (~ 0.1 ms)
Narrow field-of-view: forward scattered photons escape
Wide field-of-view: forward scattered photons may be returned
Ice cloud
Molecules
Liquid cloud
Aerosol
Hogan (Applied Optics, 2006). Code:

10. Wide-angle multiple scattering
CloudSat multiple scattering
To extend to precip, need to model radar multiple scattering
Talk on Wednesday, session B!
New model agrees well with Monte Carlo

11. Radiance forward model
MODIS and CALIPSO each have 3 thermal infrared channels in the atmospheric window region
Radiance depends on vertical distribution of microphysical properties
Single channel: information on extinction near cloud top
Pair of channels: ice particle size information near cloud top
Radiance model uses the 2-stream source function method
Efficient yet sufficiently accurate method that includes scattering
Provides important constraint for ice clouds detected only by lidar
Ice single-scatter properties from Anthony Baran’s aggregate model
Correlated-k-distribution for gaseous absorption (from David Donovan and Seiji Kato)
MODIS solar channels provide an estimate of optical depth
Only available in daylight
Likely to be degraded by 3D radiative transfer effects
Only usable when no liquid clouds in profile … currently not used

12. Ice cloud: non-variational retrieval
Aircraft-simulated profiles with noise (from Hogan et al. 2006)
Donovan et al. (2000)
Observations
State variables
Derived variables
Retrieval is accurate but not perfectly stable where lidar loses signal
Donovan et al. (2000) algorithm can only be applied where both lidar and radar have signal

13. Variational radar/lidar retrieval
Observations
State variables
Derived variables
Lidar noise matched by retrieval
Noise feeds through to other variables
Noise in lidar backscatter feeds through to retrieved extinction

14. …add smoothness constraint
Observations
State variables
Derived variables
Retrieval reverts to a-priori N0
Extinction and IWC too low in radar-only region
Smoothness constraint: add a term to cost function to penalize curvature in the solution (J’ = l Si d2ai/dz2)

15. …add a-priori error correlation
Observations
State variables
Derived variables
Vertical correlation of error in N0
Extinction and IWC now more accurate
Use B (the a priori error covariance matrix) to smooth the N0 information in the vertical

16. …add visible optical depth constraint
Observations
State variables
Derived variables
Slight refinement to extinction and IWC
Integrated extinction now constrained by the MODIS-derived visible optical depth

17. …add infrared radiances
Observations
State variables
Derived variables
Poorer fit to Z at cloud top: information here now from radiances
Better fit to IWC and re at cloud top

18. Radar-only retrieval Retrieval is poorer if the lidar is not used
Observations
State variables
Derived variables
Use a priori as no other information on N0
Profile poor near cloud top: no lidar for the small crystals
Retrieval is poorer if the lidar is not used

19. Radar plus optical depth
Observations
State variables
Derived variables
Optical depth constraint distributed evenly through the cloud profile
Note that often radar will not see all the way to cloud top

20. Radar, optical depth and IR radiances
Observations
State variables
Derived variables

21. CloudSat-CALIPSO-MODIS example
Lidar observations
Radar observations
1000 km

22. CloudSat-CALIPSO-MODIS example
Lidar observations
Lidar forward model
Radar observations
Radar forward model

23. Radar-lidar retrieval
Extinction coefficient
Ice water content
Effective radius
Forward model
MODIS 10.8-mm observations

24. …add infrared radiances
Radiances matched by increasing extinction near cloud top
Forward model
MODIS 10.8-mm observations

25. Retrievals with different radar and lidar detection
Lidar observations
Lidar forward model
Radar forward model
Visible extinction
Ice water content
Effective radius
Radar observations
MODIS radiance 10.8um
Forward modelled radiance

26. One orbit in July 2006

27. Comparison with Met Office model
log10(IWC[kg m-3])
A-Train
Model
Antarctica
Central
Pacific
Arctic
Ocean
Atlantic
South
Russia

28. Effective radius versus temperature
All clouds
An effective radius parameterization?

29. Frequency of IWC vs. temperature
log10(IWC)
Radar+lidar only
Mean and variance of IWC both increase with temperature
Clearly need both radar and lidar to detect full range of ice clouds
log10(IWC [kg m-3])
log10(IWC)
Lidar only
Radar only

30. Comparison of mean effective radius
July 2006 mean value of re=3IWP/2tri from CloudSat-CALIPSO only
Just the top 500 m of cloud
MODIS/Aqua standard product

31. Comparison of ice water path
Mean of all skies
Mean of clouds
CloudSat-CALIPSO
MODIS
Need longer period than just one month (July 2006) to obtain adequate statistics from poorer sampling of radar and lidar

32. Comparison of optical depth
Mean of all skies
Mean of clouds
CloudSat-CALIPSO
MODIS
Mean optical depth from CloudSat-CALIPSO is lower than MODIS simply because CALIPSO detected many more optically thin clouds not seen by MODIS
Hence need to compare PDFs as well

33. Differences between hemispheres
No obvious differences in the general trend
IWC shifted to low temperatures in southern hemisphere
Temperature [˚C]
Temperature [˚C]
Boreal Summer
Austral Winter
Such information is crucial for global atmospheric modelling

34. Comparison with model IWC
A-Train
Met Office
ECMWF
Temperature (°C)
Temperature (°C)
Global forecast model data extracted underneath A-Train
A-Train ice water content averaged to model grid
Met Office model lacks observed variability
ECMWF model has artificial threshold for snow at around 10-4 kg m-3

35. Structure of Southern Ocean cirrus
Observations
Note limitations of each instrument
Retrievals

36. Tropical Indian Ocean cirrus
MODIS infrared window radiance
Turbulent fall-streaks in lower half of cloud?
Stratiform region in upper half of cloud?
Observations
Note limitation of each instrument
Retrievals

37. Mid-latitude cirrus Tropical cirrus
1300 km
320 km
600 km
120 km
Stratiform upper region dominated by larger scales
A-Train data show quite different structure above ~12.5 km in tropical cirrus: gravity waves?
Outer scale 90 km
-5/3 law
Hogan and Kew (QJ 2005) found that mid-latitude cirrus structure affected by cloud top turbulence with a typical outer scale of km

39. Summary and future work
New dataset provides a unique perspective on global ice clouds
Planned retrieval enhancements
Retrieve liquid clouds and precipitation at the same time to provide a truly seamless retrieval from the thinnest to the thickest clouds
Incorporate microwave and visible radiances
Adapt for EarthCARE satellite (ESA/JAXA: launch 2013)
Model evaluation
How can Met Office and ECMWF model cloud schemes be improved?
High-resolution simulations of tropical convection in “CASCADE”
Use CERES to determine the radiative error associated with misrepresented clouds in model
Cloud structure and microphysics
What is the explanation for the different regions in tropical cirrus?
What determines the outer scale of variability?
Can we represent tropical cirrus in the Hogan & Kew fractal model?
Can we resolve the “small crystal” controversy?

41. Convergence
The solution generally converges after two or three iterations
When formulated in terms of ln(a), ln(b’) rather than a, b’, the forward model is much more linear so the minimum of the cost function is reached rapidly

42. Enforcing smoothness 1 Cubic-spline basis functions
Let state vector x contain the amplitudes of a set of basis functions
Cubic splines ensure that the solution is continuous in itself and its first and second derivatives
Fewer elements in x  more efficient!
Forward model
Convert state vector to high resolution: xhr=Wx
Predict measurements y and high-resolution Jacobian Hhr from xhr using forward model H(xhr)
Convert Jacobian to low resolution: H=HhrW
Representing a 50-point function by 10 control points
The weighting matrix

43. Enforcing smoothness 2
Twomey matrix, for when we have no useful a priori information
Add a term to the cost function to penalize curvature in the solution: ld2x/dr2 (where r is range and l is a smoothing coefficient)
Implemented by adding “Twomey” matrix T to the matrix equations