1. Assimilating satellite cloud information with an Ensemble Kalman Filter at the convective scale Annika Schomburg, Christoph Schraff EUMETSAT fellow day, 17 March 2014, Darmstadt

2. annika.schomburg@dwd.de Motivation: Weather situation 23 October 2012 2 12 UTC synoptic situation: stable high pressure system over central Europe

3. annika.schomburg@dwd.de Motivation: Weather situation 23 October 2012 3 12 UTC synoptic situation: low stratus clouds over Germany Satellite cloud type classification

4. annika.schomburg@dwd.de Motivation: Verification for 23 October 2012 4 12 hour forecast from 0:00 UTC Low cloud cover: COSMO- DE versus satellite Total cloud cover: COSMO-DE versus synop T2m: COSMO-DE minus synop Green: hits; black: misses red: false alarms, blue: no obs Courtesy of K. Stephan

5. annika.schomburg@dwd.de Motivation  Main motivation: improve cloud cover simulation of low stratus clouds in stable wintertime high-pressure systems May also be useful for frontal system or convective situations If convective clouds are captured better while developing, convective precipitation may be improved 5

6. annika.schomburg@dwd.de Introduction The COSMO model The ensemble Kalman filter Outline 6 Cloud data Assimilation approach Assimilated variables and model equivalents Results Single observation experiments Cycling experiment Forecast Conclusion and outlook Future application

7. annika.schomburg@dwd.de Introduction The COSMO model The ensemble Kalman filter Outline 7 Cloud data Assimilation approach Assimilated variables and model equivalents Results Single observation experiments Cycling experiment Forecast Conclusion and outlook Future application

8. annika.schomburg@dwd.de The COSMO model 8 COSMO-DE : Limited-area short-range numerical model weather prediction model  x  2.8 km / 50 vertical layers Explicit deep convection  New data assimilation system : Implementation of the Ensemble Kalman Filter

9. annika.schomburg@dwd.de Introduction The COSMO model The ensemble Kalman filter Outline 9 Cloud data Assimilation approach Assimilated variables and model equivalents Results Single observation experiments Cycling experiment Forecast Conclusion and outlook Future application

10. annika.schomburg@dwd.de 10 Local Ensemble Transform Kalman Filter LETKF Analysis perturbations: linear combination of background perturbations Obs First guess ensemble members are weighted according to their departure from the observations. OBS-FG R Background error covariance Observation errors

11. annika.schomburg@dwd.de 11 Local Ensemble Transform Kalman Filter LETKF Analysis perturbations: linear combination of background perturbations Obs OBS-FG R Observation errors 11 Local: the linear combination is fitted in a local region - observation have a spatially limited influence region Transform: most computations are carried out in ensemble space  computationally efficient  Implementation after Hunt et al., 2007 Background error covariance

12. annika.schomburg@dwd.de Local Ensemble Transform Kalman Filter LETKF Analysis perturbations: linear combination of background perturbations Obs OBS-FG R Background error covariance Observation errors Additional: one deterministic run: Kalman gain matrix from LETKF 12

13. annika.schomburg@dwd.de Introduction The COSMO model The ensemble Kalman filter Outline 13 Cloud data Assimilation approach Assimilated variables and model equivalents Results Single observation experiments Cycling experiment Forecast Conclusion and outlook Future application

14. annika.schomburg@dwd.de Which observations?  For shortrange limited-area models: geostationary satellite data: Meteosat-SEVIRI (Δx ~ 5km over central Europe, Δt=15 min)  Here: Assimilation of NWC-SAF cloud top height Source: EUMETSAT Height [km] Cloud top height 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Retrieval algorithm needs temperature and humidity profile information from a NWP model  cloud top height might be at wrong height if temperature- profile in NWP model is not simulated correctly!  use also radiosonde information where available

15. annika.schomburg@dwd.de 15 “Cloud analysis“: Combine satellite & radiosonde information Use nearby radiosondes within the same cloud type to correct (or approve) cloud top height from satellite cloud height retrieval Satellite cloud top Radiosonde relative humidity Radiosondes: coverage Satellite cloud type

16. annika.schomburg@dwd.de 16 Combine satellite & radiosonde information: data availability flag Use temporal and spatial distance of radiosonde for weighting: Also use data availability flag for observation error specification: 1.0 0.8 0.6 0.4 0.2 γ

17. annika.schomburg@dwd.de 17 Combine satellite & radiosonde information Satellite cloud productCloud analysis Cloud top height Obs-error [m]

18. annika.schomburg@dwd.de Introduction The COSMO model The ensemble Kalman filter Outline 18 Cloud data Assimilation approach Assimilated variables and model equivalents Results Single observation experiments Cycling experiment Forecast Conclusion and outlook Future application

19. annika.schomburg@dwd.de Determine the model equivalent cloud top  Avoid strong penalizing of members which are dry at CTH obs but have a cloud or even only high humidity close to CTH obs  search in a vertical range  h max around CTH obs for a ‘best fitting’ model level k, i.e. with minimum ‘distance’ d: relative humidity height of model level k = 1  use y= CTH obs H(x)=h k and y= RH obs =1 H(x)=RH k ( relative humidity over water/ice depending on temperature) as 2 separate variables assimilated by LETKF  use y= CTH obs H(x)=h k and y= RH obs =1 H(x)=RH k ( relative humidity over water/ice depending on temperature) as 2 separate variables assimilated by LETKF 19 Z [km] RH [%] CTH obs k1k1 k2k2 k3k3 k4k4 k5k5 Cloud top model profile (make sure to choose the top of the detected cloud )

20. annika.schomburg@dwd.de 20 Example: 17 Nov 2011, 6:00 UTC Observations and model equivalents RH model level k Observation Model „Cloud top height“

21. annika.schomburg@dwd.de 3 6 9 12 Z [km] „no high cloud“ „no mid-level cloud“ „no low cloud“ CLC assimilate cloud fraction CLC = 0 separately for high, medium, low clouds model equivalent: maximum CLC within vertical range What information can we assimilate for pixels which are observed to be cloudfree? Determine model equivalent: cloudfree pixels 21

22. annika.schomburg@dwd.de 22  COSMO cloud cover where observations “cloudfree” Example: 17 Nov 2011, 6:00 UTC High clouds (oktas)Mid-level clouds (oktas)Low clouds (oktas)

23. annika.schomburg@dwd.de Schematic illustration of approach 23 cloudy obs cloud-free obs = 1 = 0

24. annika.schomburg@dwd.de Introduction The COSMO model The ensemble Kalman filter Outline 24 Cloud data Assimilation approach Assimilated variables and model equivalents Results Single observation experiments Cycling experiment Forecast Conclusion and outlook Future application

25. annika.schomburg@dwd.de 25 „Single observation“ experiment Analysis for 17 November 2011, 6:00 UTC (no cycling) Each column is affected by only one satellite observation Objective: –Understand in detail what the filter does with such special observation types –Does it work at all? –Detailed evaluation of effect on atmospheric profiles –Sensitivity to settings

26. annika.schomburg@dwd.de relative humidity cloud cover cloud water cloud ice observed cloud top 3 lines in one colour indicate ensemble mean and mean +/- spread 1 analysis step, 17 Nov. 2011, 6 UTC (wintertime low stratus) vertical profiles Single-observation experiments: missed cloud event 26

27. annika.schomburg@dwd.de observed cloud top observation location specific water content [g/kg] relative humidity [%] Cross section of analysis increments for ensemble mean Single-observation experiments: missed cloud event 27

28. annika.schomburg@dwd.de 28 Deterministic run Humidity at cloud layer is increased in deterministic run Relative humidity Cloud cover Cloud water Cloud ice Observed cloud top First guessAnalysis

29. annika.schomburg@dwd.de Missed cloud case: Effect on temperature profile temperature profile [K] (mean +/- spread) first guess analysis  LETKF introduces inversion due to RH  T cross correlations in first guess ensemble perturbations  LETKF introduces inversion due to RH  T cross correlations in first guess ensemble perturbations observed cloud top 29

30. annika.schomburg@dwd.de relative humidity cloud cover cloud water cloud ice observed cloud top 3 lines on one colour indicate ensemble mean and mean +/- spread vertical profiles  assimilated quantity: cloud fraction (= 0) Single-observation experiments: False alarm cloud 30

31. annika.schomburg@dwd.de 31 Increment cross section ensemble mean Observed cloud top Observation location

32. annika.schomburg@dwd.de Observation cloudfree  assimilated quantity: cloud fraction (= 0) FG ANA low cloud cover fraction [octa]mid-level cloud cover fraction [octa] Observation minus model histogram over ensemble members FG ANA Single-observation experiments: False alarm cloud  LETKF decreases erroneous cloud cover despite very non-Gaussian distributions 32

33. annika.schomburg@dwd.de Introduction The COSMO model The ensemble Kalman filter Outline 33 Cloud data Assimilation approach Assimilated variables and model equivalents Results Single observation experiments Cycling experiment Forecast Conclusion and outlook Future application

34. annika.schomburg@dwd.de 1-hourly cycling over 21 hours with 40 members 13 Nov., 21UTC – 14 Nov. 2011, 18UTC wintertime low stratus Thinning: 8 km 14 km 20 km Sensitivity experiment: Data thinning 34

35. annika.schomburg@dwd.de Sensitivity experiment: Data density  Comparing experiments with different data density: 8 km 14 km 20km 35  For cloudy pixels best results are obtained for a 14 km thinning distance, for cloud-free observations no clear conclusion  For cloudy pixels best results are obtained for a 14 km thinning distance, for cloud-free observations no clear conclusion RMSE and bias averaged over all cloudy observations RMSE Bias (OBS-FG) Mean squared error for low/medium/high cloud cover averaged over all observed cloud free pixels Low clouds Mid-level clouds High clouds Solid: 8km Dashed: 14km Dotted: 20km RH at observed cloud topCloud cover

36. annika.schomburg@dwd.de  The spread for the 8km thinning experiment is lower than the other two, the difference in spread between the 14 and 20 experiments is smaller.  Ensemble is underdispersive, but there is no sign of a further reduction of the spread during the cycling  The spread for the 8km thinning experiment is lower than the other two, the difference in spread between the 14 and 20 experiments is smaller.  Ensemble is underdispersive, but there is no sign of a further reduction of the spread during the cycling 36  Comparing experiments with different data density: 8 km 14 km 20km Sensitivity experiment: Data density RMSE Spread

37. annika.schomburg@dwd.de 1-hourly cycling over 21 hours with 40 members 13 Nov., 21UTC – 14 Nov. 2011, 18UTC Wintertime low stratus Thinning: 14 km Comparison cycling experiment: only conventional vs conventional + cloud data 37

38. annika.schomburg@dwd.de Time series of first guess errors, averaged over cloudy obs locations assimilation of conventional obs only assimilation of conventional + cloud obs RMSE Bias (OBS-FG) 38 Comparison “only conventional“ versus “conventional + cloud obs"  Cloud assimilation reduces RH (1-hour forecast) errors RH (relative humidity) at observed cloud top

39. annika.schomburg@dwd.de 39 conventional only conventional + cloud Total cloud cover of first guess fields after 20 hours of cycling Satellite cloud top height Comparison of cycled experiments satellite obs 12 Nov 2011 17:00 UTC

40. annika.schomburg@dwd.de Time series of first guess errors, averaged over cloud-free obs locations (errors are due to false alarm clouds) mean square error of cloud fraction [octa]  False alarm clouds reduced through cloud data assimilation Cycled assimilation of dense observations 40 low clouds High clouds

41. annika.schomburg@dwd.de low clouds mid-level clouds high clouds ‘false alarm’ cloud cover (after 20 hrs cycling) conventional + cloud conventional obs only 41 Comparison “only conventional“ versus “conventional + cloud obs" [octa]

42. annika.schomburg@dwd.de Introduction The COSMO model The ensemble Kalman filter Outline 42 Cloud data Assimilation approach Assimilated variables and model equivalents Results Single observation experiments Cycling experiment Forecast Conclusion and outlook Future application

43. annika.schomburg@dwd.de 24h deterministic forecast based on analysis of two experiments (after 12 hours of cycling) 14 Nov., 9UTC – 15 Nov. 2011, 9UTC Wintertime low stratus Comparison forecast experiment: only conventional vs conventional + cloud data 43

44. annika.schomburg@dwd.de  The forecast of cloud characteristics can be improved through the assimilation of the cloud information 44 Comparison of free forecast: time series of errors Conventional + cloud data Only conventional data RMSE Bias (Obs-Model) Low clouds Mid-level clouds High clouds Mean squared error averaged over all cloud-free observations RH (relative humidity) at observed cloud top averaged over all cloudy observations

45. annika.schomburg@dwd.de Verification against independent measurements Errors for SEVIRI infrared brightness temperatures (model values computed with RTTOV) 45  RMSE is smaller for first 16 hours of forecast for cloud experiment, bias varies RMSE Bias (Obs-Model) Conventional + cloud data Only conventional data

46. annika.schomburg@dwd.de CONV+CLOUD experiment Only CONV experiment  Also the high clouds are simulated better in the cloud experiment 46 Verification against independent measurements: SEVIRI brightness temperature errors Cloud top height 14 Nov 2011, 18 UTC

47. annika.schomburg@dwd.de Introduction The COSMO model The ensemble Kalman filter Outline 47 Cloud data Assimilation approach Assimilated variables and model equivalents Results Single observation experiments Cycling experiment Forecast Conclusion and outlook Future application

48. annika.schomburg@dwd.de Use of (SEVIRI-based) cloud observations in LETKF: Tends to introduce humidity / cloud where it should (+ temperature inversion) Tends to reduce ‘false-alarm’ clouds Despite non-Gaussian pdf’s Long-lasting free forecast impact for a stable wintertime high pressure system Conclusions 48

49. annika.schomburg@dwd.de Evaluate impact on other variables (temperature, wind) Other cases (e.g. convective) Also work on direct SEVIRI radiance assimilation Revision on QJRMS article on single observation experiments, publish second article on full cycling and forecasts results Application in renewable energy project EWeLiNE… Next steps 49

50. annika.schomburg@dwd.de Introduction The COSMO model The ensemble Kalman filter Outline 50 Cloud data Assimilation approach Assimilated variables and model equivalents Results Single observation experiments Cycling experiment Forecast Conclusion and outlook Future application

51. annika.schomburg@dwd.de Renewable energy project Germany plans to increase the percentage of renewable energy to 35% in 2020  Increasing demands for accurate power predictions for a safe and cost-effective power system  Joint project of DWD and Fraunhofer-Institut for Wind Energy & Energy System Technology in Kassel and three transmission network operators EWeLiNE Objective: improve weather and power forecasts for wind and phovoltaic power and develop new prognostic products 4 year project, 13 researchers at DWD

52. annika.schomburg@dwd.de Problematic weather situations for photovoltaic power prediction  Cloud cover after cold front pass  Convective situations  Low stratus / fog weather situations  Snow coverage of photovoltaic modules 52

53. annika.schomburg@dwd.de Problematic weather situations for photovoltaic power prediction  Cloud cover after cold front pass  Convective situations  Low stratus / fog weather situations  Snow coverage of photovoltaic modules 53

54. annika.schomburg@dwd.de Problematic weather situations for photovoltaic power prediction  Cloud cover after cold front pass  Convective situations  Low stratus / fog weather situations  Snow coverage of photovoltaic modules 54

55. annika.schomburg@dwd.de Hochrechnung Example: problematic weather situation: low stratus clouds Error Day-Ahead: 4800 MW Low stratus clouds not predicted Low stratus clouds observed in reality Projection Day-Ahead Intra-Day time Power from PV modules courtesy by TENNET 55

56. annika.schomburg@dwd.de Thank you for your attention! Thanks to EUMETSAT for funding this fellowship!