1. Page 1© Crown copyright 2004 1D-VAR Retrieval of Temperature and Humidity Profiles from Ground-based Microwave Radiometers Tim Hewison and Catherine Gaffard MicroRad’06 2 March 2006, Puerto Rico

2. Page 2© Crown copyright 2004 Contents  Principles of Bayesian Inversion  Requirements:  Background data and choice of state vector  Observations and error characteristics  Forward Model and its Jacobian  Expected Performance  Minimisation  Example Retrievals  Cloud Classification Scheme  Statistics of Retrievals  Conclusions & Future Work

3. Page 3© Crown copyright 2004 Principals of Bayesian Inversion  Determine most probable state vector, x a, given  Observation, y  Knowing error characteristics of each (assumed Gaussian)  Minimise cost function: J(x a )= (y-H(x b )) T R -1 (y-H(x b )) + (x a - x b ) T B -1 (x a - x b )  Requires:  Observations, y (MW radiometer, surface, IR, ceilometer, …)  Background, x b  Background Error Covariance Matrix, B  Observation Error Covariance Matrix, R  Observation Operator, H(x)  Jacobian of H(x), H(x)=  x H(x)=  y/  x

4. Page 4© Crown copyright 2004 Background and State Vector, x  Need background, x b to resolve ill-posed problem  Use T+3 to T+9 Forecast  Independent of validation  28 lowest levels  Concentrated near surface  T(z), q(z), L(z)  Fix profile above 14km  Choice of State Vector: x=[T 1,..T 28, lnq t1,.. lnq t28 ] 17 Sites of archived profiles from UK Mesoscale model UMG3 Models levels Met Office UK Mesoscale Model

5. Page 5© Crown copyright 2004 Total Water control variable, ln(q t )  Use Total Water (excl. ice), ln(q t )=ln(q+q L )  Errors more Gaussian than q  Implicit super-saturation constraint  Enforces q-q L correlation  Reduces size of x (and H)  Deblonde and English, 2003:  Linear Partition Function  Discontinuities in dq L /dq t  Hewison [2005]:  Smooth Partition Function Total Water partition functions by Deblonde and English (dashed) and Hewison (solid) for a range of q t =(0 – 1.2)q sat. Upper panel: q (black) and liquid water mixing ratio, q L (blue). Middle panel: derivatives dq/dq t (black) and dq L /dq t (blue). Lower panel: 30 GHz absorption

6. Page 6© Crown copyright 2004 Background Error Covariance Matrix  Background Error Covariance Matrix, B  Used for satellite data  T and ln(q)  Assume B(T,lnq t )=B(T,lnq)  T-q terms=0  Also possible to calculate from T+6 F/C + Sondes B matrix for 1D-VAR of ATOVS (30N-90N) interpolated to UMG3 levels – mesospheric values are omitted. Bottom left: temperature covariances [K 2 ]. Upper right: specific humidity covariances [ln q(kg/kg) 2 ]. T level q level

7. Page 7© Crown copyright 2004 Radiometrics TP/WVP-3000 Microwave Radiometer  7 Channels: 51-59 GHz  O 2 band - temp. profile  5 Channels: 22-30 GHz  H 2 O line - humidity, cloud  Pressure, temp., RH sensors  Dew Blower & Rain Sensor  Infrared Radiometer  Cloud base temperature  Automatic Calibration  black body, noise diode  Zenith and Elevation Scans  Observation Cycle: ~1 min Radiometrics MP3000 Microwave Radiometer at Camborne

8. Page 8© Crown copyright 2004 Estimating Observation Error Covariance Matrix  Observation Error Covariance Matrix, R, includes: R=E+F+M  Radiometric Noise, E  Forward Model Error,F  Representativeness Error, M (sub-grid var.)  Evaluate each term  M dominates M>F>E  But varies by factor of >10  So calculate dynamically! Total Observation Error covariance matrix, R First 12 channels are Radiometrics MP3000 Tb [K 2 ], Last 2 channels are ambient temp [K 2 ], RH [lnq] Channel Number  Obs. vector, y=[T b1, T b2, … T b12, T amb, lnq amb, T ir ]

9. Page 9© Crown copyright 2004 Forward Model and its Jacobian  Radiative Transfer Model  Rosenkranz’98 for microwave  1:1 for surface sensors  Finite extinctions for infrared  Calc. Jacobian, H=H’(x)=  y/  x  By ‘brute force’ – Slow!  Perturb x by  x=[1 K,0.001]  Check Linearity |y(x+  x) -y(x-  x)|<<diag(E)  Equivalent monochromatic frequencies for microwave  Fast Model (“FAP”)  Polynomial fit  to p, T, q  Could modify RTTOV Jacobians for the Radiometrics TP/WVP-3000 V-band channels to temperature perturbations of 1K, scaled by the level spacing

10. Page 10© Crown copyright 2004 Error Analysis of Retrieved Profiles  Gaussian linear case: Analysis error of optimal estimation retrieval: A = (H T R -1 H + B -1 ) -1  Compare with B:  T: A < B for z<5km,  T<1 K q: A < B for z<3km,  lnq<0.4  For q, A depends on state  Using surface sensors only – A < B for z<500m  A ~ sondes for z<1km Background Error, B, (black) and Analysis Error, A, using Radiometer (red), Only surface sensors (green), Radiosonde (blue).

11. Page 11© Crown copyright 2004 Vertical Resolution  Gain Matrix, K = BH T (HBH T +R) -1  Averaging Kernel Matrix = KH  Vertical Resolution of Analysis,  z.diag((KH) -1 )  ~2x larger than other def n s  T profile resolution increases with height ~2z  lnq profile resolution =  (x)  Some q resolution for z<1km, but IWV above Vertical resolution of analysis temperature and humidity (lnq t ) profiles calculated as the inverse of the trace of the averaging kernel matrix [Purser and Huang, 1993] (US Std Atm)

12. Page 12© Crown copyright 2004 Optimal Estimation Iterative Retrieval  Can minimise linear problems analytically  But humidity retrieval is moderately non-linear - use Gauss-Newton iteration: x i+1 =x i +(B -1 +H i T RH i ) -1 [H i T R -1 (y o -H(x i ))-B -1 (x i -x b )]  Test for convergence: [H(x i+1 )-H(x i )] T S  y -1 [H(x i+1 )-H(x i )]<<m where S  y is the covariance matrix between the measurement and H(x i ) m is the dimension of y o (number of channels)  Calculate error covariance of analysis, A = (H T R -1 H + B -1 ) -1  Test  2 of result

13. Page 13© Crown copyright 2004 Example retrievals  100 synthetic observations, y o based on real sonde, x t and NWP background, x b  Forecast inversion too low  Overestimated the humidity x2  83% converged in ~9 iterations on average  Retrievals closely clustered  Robust to observation noise  Retrievals closer to x t than x b  Thins the cloud  B makes it impossible for retrieval to move inversion Retrievals (red), Background (black), Radiosonde (blue). Left panel shows temperature profiles. Right panel shows profiles of humidity (lnq) and liquid water (lnq l ) and specific humidity at saturation (dotted)

14. Page 14© Crown copyright 2004 Cloud Classification  Convergence poor in cloudy profiles  Residuals non-Gaussian  New cloud classification:  T ir  T amb -40  Clear  Use lnq  Supersaturation penalty  T ir >T amb -40  Cloudy  Use lnq T as before  Add T ir to y Statistics of 1D-VAR retrievals using synthetic observations and background. 77 cloudy cases Camborne, UK, Winter 2004/05. Solid lines – retrievals and background SD. Dashed lines – bias. Error covariances diagonals – dotted lines for the analysis, A, Black lines for the background, B. Red lines show the statistics of the cloudy 1D-VAR retrieval.

15. Page 15© Crown copyright 2004 Conclusions  Pros  Optimal method to integrate observations with background  Provides estimate of error in retrieval  Shows impact from MWR below ~4km – most <1km  Cons  Convergence problems for very non-linear problems  Difficult when background is wrong (shifting patterns)  Future Work  Add ceilometer cloud base/cloud radar tops/GPS IWV to y  Integrate with Wind Profiler SNR – e.g. Boundary Layer top  How to exploit high time resolution? 4D-VAR? Variability?

16. Page 16© Crown copyright 2004 Thank You! Any Questions?