1. Data Assimilation in Hydrology
University of Colorado Boulder
Data Assimilation in Hydrology
Andrew Slater, NSIDC/CIRES

2. Presentation Outline Data Assimilation – the concept & steps
Data Assimilation Methods – Simple to Complex
Theory compared to reality – the issues!
The Next Step …..

3. Uncertainty in Numerical Modeling
(1)
Model Structure
Parameterizations & components
Numerical methods
Model Forcing
Spatial & Temporal structure
Parameter Data
Soils & Vegetation, type and distribution
Initial Conditions
Influences trajectory (forecasting = IVP)
(2)
(3)
(4)

4. Data Assimilation : the concept
The objective of Assimilation is to characterize
the state of a system
Involves combining models and observations
Question : How to do this best?

5. Assimilation vs. Calibration
Occurs during the simulation
Usually applied to model states
Calibration -
Post-simulation adjustment
Usually applied to model parameters
Both are important in hydrologic modeling

6. Operational Streamflow Forecasting Method
Historical Data
Forecasts
SNOW-17 / SAC
Historical Simulation
SWE SM Q
Past
Future
Run hydrologic model up to the start of the forecast period to estimate basin initial conditions;

7. Operational Streamflow Forecasting Method
Historical Data
Forecasts
SNOW-17 / SAC
SNOW-17 / SAC
Historical Simulation
SWE SM Q
Past
Future
Run hydrologic model up to the start of the forecast period to estimate basin initial conditions;
Run hydrologic model into the future, using an ensemble of local-scale weather and climate forecasts.

8. Data Assimilation: The Basics
Improve knowledge of Initial conditions
Assimilate observations at time t
Model “relocated” to new position

9. Assimilation Methods All models & observations have errors! Rule Based
Direct Insertion
Nudging / Relaxation
Optimal Interpolation
3D-Var
Kalman Filtering
4D-Var
Kalman Filter/Smoother

10. Rule Based Assimilation
Model states and observations do not equate
Example – snow covered area from a satellite says nothing about the amount of SWE
Rule: If SCAobs > 0 and SWEmod = 0
SWE+mod = SWE-mod + 20mm
Blunt, brutal and involves lots of assumptions!

11. Example: Direct Insertion & Nudging
SNOTEL x
Small basin with SNOTEL type station
Objective : determine basin SWE
Observation is SWE, as is model state
DI : Assumes observation is perfect
NN: Nudges model as suggested by observation

12. Direct Insertion Assimilation

13. Newtonian Nudging Assimilation

14. Optimized Assimilation: General Case
Predict model states (X)
Get relative weights (K) of model and observations
Update model state as a combination of its own projected state and that of the observations (z)
P = model error
R = observation error
Xt- = AXt-1 + Bft
Kt = P(P + R)-1
Xt+ = Xt- + Kt(zt – Xt- )

15. Optimized Assimilation : Scalar Example
Our Model predicts : X- = 6
Model error variance : P = s2x = 2

16. Optimized Assimilation : Scalar Example
Our Observations say : Z = 4
Obs. error variance : R = s2z= 1

17. Optimized Assimilation : Scalar Example
Combined Model and Observations say :
X+ = 6 + (2/(2+1)) x (4 – 6)
Our Analysis is X+ = 4.66
Analysis variance : s2a= 0.66
Analysis Variance

18. Optimized Assimilation: OI & 3D-Var
Optimal Interpolation ≈ 3D-Var
Different way of solving same problem
So far we assume the errors are known
Efficient, simple
Many issues unresolved … We continue.

19. The Kalman Filter (Linear Case)
Predicting model error (P)
Equating model states to observed items (H)
Recursive filter
Xt- = AXt-1 + Bft
Kt = PtHT(HPtHT + R)-1
Xt+ = Xt- + Kt(zt – HXt- )
Pt+ = (I – KH)Pt-
Pt+1- = APt+AT+ Qt
6. Xt+1- = AXt + Bft+1

20. PROBLEMS: How do we ... Equate observations and model states (H)?
Relate model states to each other?
Assess model error (P)?
Assess observation error (R)?
“Balance” our model? (so it doesn’t blow up)
Account for time differences?

21. Equating Observation & Model State: (H)
Location problem
External Interpolation
Assess through Filter
Retrieval problem (Measurement model)
Using satellite retrievals e.g. SWE product
Convert radiances to model states
Relating model states
If we update state A, what to do with state B ?
Will our model freak out?

22. Kriging
With tuned
variogram
Obs.
Barnes
Inverse
Distance

23. Case Study: Full energy balance 2 Tile surface Bare soil Vegetated
5 Thermal Layers
Desborough, (1999)

24. Case Study: Assimilate this temperature Low soil resolution
Composite snow
Coupled moisture
30 minute timestep
Assimilate once every 3 days 25

25. Real Data : Seward Peninsula, AK
ATLAS Data

26. The Ensemble Kalman Filter CHASM Soil Updating System
X = 7 State variables used in CHASM
T2, TSurf, T3, TT1, TT2, SWE, Density
Observations are “pre-transformed” via interpolation
Xt- = AXt-1 + Bft
Kt = PtHT(HPtHT + R)-1
Xt+ = Xt- + Kt(zt – HXt- )

27. Control Simulation
25 cm
Council – Model
Council – Obs.

28. The Ensemble Kalman Filter
25 cm
Council – Model
Council – Obs.

29. Control Simulation
70 cm
Council – Model
Council – Obs.

30. Direct Insertion of 25cm Soil Temperature
Council – Model
Council – Obs.
Diffusion propagates information

31. Ensemble Kalman Filter
70 cm
Council – Model
Council – Obs.
Kalman Filter propagates information

32. Now Propagate Information
Update Kougarok based on Council
No Observations from Kougarok used
Cannot Interpolate observations (no data)
Expand the state variable vector
Exploit the covariance relationship
No parameter changed

33. Control Simulation
25 cm
Council : Model
Council : Obs.
Kougarok : Model
Kougarok : Obs
Next : Only the Council 25cm observations are assimilated but we propagate the information

34. Ensemble Kalman Filter
70 cm
Council : Model
Council : Obs.
Kougarok : Model
Kougarok : Obs

35. Assessing Model Error (P)
Adjoint Model
Propagates error through derivatives
A pain to code
Assumes Gaussian error
Ensemble Method (EnKF)
Gets covariance between ensemble members
No assumption on error structure
Easy to code

36. Generating an Ensemble
(2-km grid—150 x 150 pixels)
Estimate precipitation
CDF at each grid cell
Synthesize ensembles
from the CDF
Conditional CDF
(ord. least squares)
Cumulative Prob.
1-POP
(logistic regression)
Occurrence:
bnew = bold + (XTWVX)–1 XTW(Y-p)
Amounts:
b = (XTWX)–1 XTWY
Precipitation

37. POP & PCP Location: Colorado
Spatial fields of POP and Precip. (in Z-space)
Applied Logistic & OLS regression
All estimates are locally-weighted
SWE computed similarly
Clark & Slater, 2006

38. Quantify uncertainty using multiple models…
SOILR
GFLWR
GWATR
IZONE
LAYR2
LAYR1
PCTIM
ADIMP
ADIMC
UZTWC
UZFWC
LZTWC
LZFSC
LZFPC
IMPZR
RZONE
IFLWR
LZONE
PRMS
SACRAMENTO
ARNO/VIC
TOPMODEL

39. Observation Error Estimation (R)
Measurement error
Representation error
Cross Validation
For Interpolation or Retrieval
Compare Estimate to Observation

40. 53 Upper C.R.B. SNOTEL Stations

41. EnKF Example: Snow Assimilation
NWS SNOW-17 model
Generated cross validated ensemble forcing
Used cross validated observation ‘estimates’
Withholding experiments
Accounted for filter divergence
Assimilation shown to produce better results

42. EnKF Example: Snow Assimilation
Interpolated SWE Mean & Std. Dev
Model
Truth

43. White without Red = B.L.U.E
SWE contains red (time correlated) noise
Only want to use “new” information
Example – same timestep
Filter Divergence = potential problem
Slater & Clark, 2006

44. Final Assimilation Results

45. Example 2: Assimilation at Multiple Sites

46. Improved Simulations at Different Sites?
small improvement at outlet
1) assimilate at the outlet, test at interior locations
errors of opposing sign
2) assimilate at interior locations
degrade simulations
Assimilate at outlet only
compare a and b: the filter works at Barnetts Bank, but can cause degradation at other sites, and can’t fix timing error
2. Assimilate all the interior sites, but not the outlet
Compare a and c. filter has improved all 3 u/s sites, and the d/s site
can’t fix timing errors

47. Horizontal Transfer of Information
increase peak
Branch only
Assimilate only the Branch data
Filter has worked at Branch, and has raised peaks for 2 other sites
Our main motivation for this is PUB – horizontal transfer between non-nested basins

48. Horizontal Transfer of Information
increase peak
Assimilate Dip Flat only
Look at 2nd peak. Filter has improved Dip Flat, but now overestimates 2 other sites
Underlying causes include
Actual spatial correlations being weaker than those we have modelled
Parameters assumed too constant in space
Choice of 100km correlation length
Dip Flat only
Clark et al. (in press) AdWR

49. Summary Many assimilation methods available
Theory is easy, implementation is difficult
Issues:
Deriving ‘ideal’ observations
Location & Error estimation
Equating to model state variables
Model Error quantification
Ensure all areas of uncertainty are accounted for
Relating model states & maintaining model balance
Filter divergence

50. The Quest What is your quest? We seek the Holy Grail!
To successfully assimilate
cryospheric satellite data into
numerical models
Python et al, 1974

51. Satellite Data Issues Matching model needs
Validation and Error estimation
Radiances vs. Products
MODIS and AMSR-E

52. MODIS Snow Covered Area
“MOD10x … designed for use as boundary conditions into global-scale climate models.” Klein, Hall & Riggs (1998)
Bin based algorithm
Not necessarily comparable to model needs %

53. Seward Peninsula Vegetation
MODIS
AVHRR
UMD 1x1km global vegetation map
MODIS Area K C

54. 100% Snow Covered Area vs. Albedo (MODIS)
Climate models more interested in albedo
Valid boxes out of 2144
Black Sky Albedo (%) White Sky Albedo (%)

55. MODIS in the West
Yampa Basin, Colorado
Missing
Cloud
Snow
Snow-Free

56. MODIS in the West
Yampa Basin, Colorado
Missing
Cloud
Snow
Snow-Free

57. MODIS in the West Important period often cloud contaminated
No mass information included (?)
Calibration potential
SWE inversion?
Missing
Cloud
Snow
Snow-Free

58. AMSR-E – Microwave Miracles?
Radiances vs. Products
Products tend to be “global”
Statistical vs. Physical inversion
Same old questions:
Validation
Error estimate

59. AMSR-E Some information exists – can we exploit it?
Global algorithm (Chang) is not ideal

60. ERA-40 Optimal Interpolation

61. Theory!

62. Surface Energy & Mass Balance Model
Rsw
Rlw E H
esTsurf4
Precip.
Evap
Surface melt
& refreezing G
Tsurf
SWE a
Tsnow
Melt

63. Aerodynamic Transfer

64. Albedo Parameterization

65. Thermal Conductivity / Diffusivity
Tair = -10oC, SWD = 500 Wm-2, RH=80%, Wind=3ms-1
End of 1 day of forcing

66. Thermal Conductivity / Diffusivity
Tair = -10oC, SWD = 500 Wm-2, RH=80%, Wind=3ms-1
End of 4 days forcing
Cumulative effect; more or less damping

67. Snow Model Intercomparison
21 Land Surface Models of varying complexity
Identical forcing data
Parameters = a priori

68. Snow Model Intercomparison
Energy balance model calibrated on RMSE
Good performance
Some deficiencies still exist

69. Snow Model Intercomparison
Temperature index model (green line)
Mean & Seasonal Amplitude melt factor
Near equal performance

70. The Jollie Basin – A Test Case

71. Scale Dependence – Control Simulation

72. Scale Dependence – Mean Elevation

73. Intra-basin Range of 1st Order Elevation

74. Remote Sensing Potential
MODIS & AVHRR daily data
“Standard” products are limited
Cloud and Snow confused
Reclassification needed
Resolution sufficient for calibration
NIWA on the job already

75. MODIS Sub-basin Fractional Snow Cover

76. SNODAS & DMIP-2 Basins Model complexity Snow Covered Area
SNODAS w/years
Photo : A. Slater

77. SNOTEL Stations near DMIP-2

78. Snow Model Complexity - Structure
Model Structure A B C D E F G
Mean Melt Factor
Temperature Threshold for Melt
Snow/Rain Criteria
Gauge Undercatch Parameter
Seasonal Amplitude of Melt Factor
Rain on Snow Melt Factor

79. Snow Model Complexity – Forcing Error

80. Snow Model Complexity - Calibration
Heavenly
Valley
Blue
Lakes
Ebbetts
Pass

81. Snow Volume and Streamflow: Carson

82. Snow Water Equivalent: Carson

83. SNODAS vs SNOTEL

84. SNODAS Water Balance

85. MODIS Assessment & Field Validation
Photo : A. Slater

86. MODIS SCA vs SNODAS SWE

87. SNODAS SWE to MODIS SCA

88. Measurement Methods Snow Water Equivalent Snow Depth Precipitation
Meteorology etc.
December 8th, 2007

89. Measurement Methods SNOTEL and Precipitation Gauges Snow Board
Photos: A. Slater

90. Measurement Methods
Sonic Snow Depth Sensor
Photos: A. Slater

91. Measurement Methods
Alter
DFIR
Nipher
Wyoming
Photos: NCAR

92. Measurement Methods
Pyranometer and Stevenson Screen
Photos: A. Slater

93. Other Data Sources Snow courses & weather networks CAIC Tower
Berthoud Pass
Snow courses & weather networks
Photos: A. Slater

94. Snow Regimes Maritime Inter-mountain Continental Deep snow (>3m)
Warmer + wetter snow
Inter-mountain
Mid-depth (1.5m-3m)
Continental
Shallow snow (<1.5m)
Cold + Low density
~7% water content in CO
Tremper, 2004

95. Snow Stratigraphy Crystal Formation Snow Metamorphism Avalanches
Equitemperature (ET)
Temperature Gradient (TG)
Avalanches
10 Mile Range, CO – Jan 2008

96. Snow Crystal Formation
Magono & Lee, 1966 (Observations)
Nakaya, 1945 (Lab Experiments)

97. Surface Hoar
Photos: A. Slater

98. Depth Hoar
Photo:

99. Avalanches – a natural hazard
Arapahoe Basin – May 2005
Photos: A. Slater

100. University of Colorado Boulder
The End
Thank You