1. © Crown copyright Met Office Does the order of the horizontal and vertical transforms matter in the representation of an operational static covariance model in global atmospheric DA? Marek Wlasak and Mike Cullen

2. Outline Give the argument for and properties of swapping the order of the vertical and horizontal transforms in the static covariance model. Show benefits of swapped order Summarise

3. © Crown copyright Met Office Argument and properties for swapping the order of the horizontal and vertical transforms

4. Static covariances are still important Hybrid DA methods rely on a static covariance model (B s ) to give sufficient degrees of freedom. Static part still contributes to over 50% of the whole forecast error. This is not going to change much in the near future unless there is a huge increase in the number of ensemble members used. (Reference: Hidden Error Variance Theory, Bishop 2013) So modelling of static covariances is important.

5. Rationale for conserving the properties of the training data We need to calibrate a covariance model with training data from an ensemble properly representing the forecast error. Ideally we would like to get to a position where the covariance model conserves some key properties and the implied background error covariance faithfully represents the training data. This philosophy should in the long run help in bootstrapping to provide a good covariance file. It also means that the emphasis moves away from tuning a covariance model, to getting the training data right.

6. Conservation of variance The total kinetic energy on each vertical level is determined by : the power spectra of the stream-function ψ’ and velocity potential χ’. ( It holds variance and gradient information.) If the power spectra of ψ’, χ’ are conserved so will the total kinetic energy. This will have a knock on effect on horizontal length scales of the balanced pressure and temperature.

7. Background covariance structure. TvThTp (swapped) Latitude dependence built into parameter transform only. Global variances and power spectra of model variables conserved. ThTvTp (current) Latitude dependence built into vertical and parameter transform. Power spectra not conserved for model variables.

8. © Crown copyright Met Office Results: Background error covariance comparison

9. Swapped order Conserve KE

11. Current order Observed (above) and Implied u (below)

12. Current order Observed (above) and Implied T (below) Not conserved for each model Level.

13. Current order Observed (above) and Implied P (below)

14. © Crown copyright Met Office Trial results

15. Non-hybrid trials Two trials were run with UM at N320 and VAR at N216 and N108. The main difference is seen with an increase in the size of the theta and pressure increments, due to the increase in associated background errors. There is also an increase horizontal length scale in pressure and theta.

16. Non-hybrid trials: Summer Case + 0.75 Severe hit on fit to analyses Model more active

17. Non-hybrid trials: Winter Case Fit to obs + 1.2

18. Summary: It is clear that swapping the transform order has a big effect on results when tested without the flow dependence of the hybrid. Neutral results have so far been obtained in hybrid trials. The biggest benefit is that the background error is now represented by the training data. It is expected that an improved choice of training data, consistent with the hybrid error modes, will help.

19. © Crown copyright Met Office Additional slides.

20. Overview of the current generation of static forecast error covariance model in the global domain. (1) Vertical transform Calibration Statistics U’ V’ P’ Θ’ ρ’ qT’ Ψ’ Χ’ Ap’ μ’ Horizontal transform A priori structure from physical equations etc Parameter transform in/out of uncorrelated variables Covariance Model Linear balance -----> Vertical Regression Hydrostatic balance _____________________ |New humidity transform| ----------------------------------- Eqn of state Vertical modes (Eigenvectors generalised) Eigenvalues Spherical harmonic basis SQRT(Power spectra) multiplied divided T transform U transform B = U U T Tp Up

21. Alternate generation of static forecast error covariance model in the global domain Calibration Statistics U’ V’ P’ Θ’ ρ’ qT’ Ψ’ Χ’ Ap’ μ’ A priori structure from Physical equations etc Parameter transform in/out of uncorrelated variables Covariance Model Spherical harmonic basis SQRT(Vertical Covariance for each total wavenumber) or inverse T transform U transform B= U U T Linear balance -----> Vertical Regression Hydrostatic balance _________________ Eqn of state New humidity transform| ---------------------------

22. Background covariance structure. The static covariance model is an approximation to the true forecast error covariance B and is constructed by making a number of prior assumptions. One choice is the order of the horizontal U h and vertical transforms (as they do not commute). i.e. (whether the vertical transform is applied in spectral space or not.) Ie. B 1 = U p U v U h (U p U v U h ) T /= U p U h U hv (U p U h U hv ) T = B 2 Where : U p is the parameter transform U h is the horizontal transform U v is the vertical transform is grid-point space. U hv is the vertical transform is done in spectral space.

23. Hybrid trials The benefit is not seen in hybrid trials Fit to observations neutral

24. Overview of the FUTURE generation of static forecast error covariance model in the global domain. (1) Calibration Statistics U’ V’ P’ Θ’ ρ’ qT’ Ψ’ Χ’ Ap’ μ’ Horizontal /vertical transform combined A priori structure from Physical equations etc Parameter transform in/out of uncorrelated variables Covariance Model PV transform Ekman balance? _____________________ |Additional balances? | ----------------------------------- Spherical harmonic basis SQRT(Vertical Covariance for each total wavenumber) or inverse. T transform U transform B= U U T Adaptive grid 1D/3D Monge – Ampère Eq

25. Basic overview. Code comprises of a number of small Fortran programs being called from ksh scripts Most important script for the user is the Launcher script. Expect all switches that the non-developer uses to be accessed from there. Glue between modules Environment variables some from Launcher NetCDF files – including its header Small text files that give the paths to groups of NetCDF files.

26. Basic overview. Code comprises of a number of small Fortran programs being called from ksh scripts Most important script for the user is the Launcher script. Expect all switches that the non-developer uses to be accessed from there. Glue between modules Environment variables some from Launcher NetCDF files – including its header Small text files that give the paths to groups of NetCDF files.

27. © Crown copyright Met Office Future Directions in Global

28. Overview of the current generation of static forecast error covariance model in the global domain. (1) Vertical transform Calibration Statistics U’ V’ P’ Θ’ ρ’ qT’ Ψ’ Χ’ Ap’ μ’ Horizontal transform A priori structure from Physical equations etc Parameter transform in/out of uncorrelated variables Covariance Model Linear balance -----> Vertical Regression Hydrostatic balance _____________________ |New humidity transform| ----------------------------------- Eqn of state Vertical modes (Eigenvectors generalised) Eigenvalues Spherical harmonic basis SQRT(Power spectra) multiplied divided. T transform U transform B = U U T Tp Up

29. To show how vertical structure is a function of horizontal total wave- number. Look at the vertical covariances of control variables for specific horizontal total wavenumber from 300 samples of unpacked ECMWF forecast differences.

30. Vertical covariances as a function of total wave- number : from 300 samples of unpacked EC data. PSI Wavenumber 1

31. Psi Wavenumber 5

32. Psi Wavenumber 10

33. Psi Wavenumber 30

34. Psi Wavenumber 100

35. Psi wavenumber 200

36. Psi wavenumber 300

37. Chi wavenumber 1

38. Chi wavenumber 5

39. Chi wavenumber 10

40. Chi wavenumber 30

41. Chi wavenumber 100

42. Csi wavenumber 200

43. Csi wavenumber 300

44. Unbalanced pressure Wavenumber 0

45. Unbalanced pressure Wavenumber 1

46. Unbalanced pressure Wavenumber 5

47. Unbalanced pressure Wavenumber 10

48. Unbalanced pressure Wavenumber 30

49. Unbalanced pressure Wavenumber 100

50. Unbalanced pressure Wavenumber 200

51. Unbalanced pressure Wavenumber 300

52. New humidity control variable Wavenumber 0

53. New humidity control variable Wavenumber 1

54. New humidity control variable Wavenumber 5

55. New humidity control variable Wavenumber 10

56. New humidity control variable Wavenumber 30

57. New humidity control variable Wavenumber 100

58. New humidity control variable Wavenumber 200

59. New humidity control variable Wavenumber 300