1. R. Reichle 1 * and C. Draper 1,2 With contributions from: G. De Lannoy, R. de Jeu, Q. Liu, V. Naeimi, R. Parinussa, W. Wagner AMSR-E Technical Interchange Meeting Oxnard, CA Sep 4-5, 2013 Soil moisture retrievals from AMSR-E and ASCAT: Error estimates and data assimilation 1 NASA/GSFC, Greenbelt, MD, USA 2 Universities Space Research Association, Columbia, MD, USA *Email:Rolf.Reichle@nasa.gov

2. Root Mean Square Error (RMSE) needed to validate against mission target accuracies, and to specify observation error for data assimilation. Difficulty: No recognized soil moisture truth at large scales.Motivation Point-scale (e.g., SCAN): Representativity errors when comparing against satellite estimates. Footprint-scale (e.g., USDA ARS): Limited spatial coverage. RMSE traditionally from validation against in situ observations:

3. Error estimates for soil moisture retrievals Triple collocation Error propagation Soil moisture data assimilation Outline

4. How can we estimate RMSE in satellite soil moisture retrievals?Approach Two methods: Triple collocation Error propagation through remote sensing retrieval algorithms Two remotely sensed soil moisture datasets: AMSR-E (X-band) Land Parameter Retrieval Model (de Jeu and Owe, 2003) ASCAT C-band TUWien empirical change detection (Wagner et al., 1999) Experiment details: Jan 2007 - Oct 2011 North American domain

5. Triple Collocation (TC) is a method to estimate RMS errors. TC requires three independent estimates. NOTE: 1.) TC cannot provide absolute RMS errors because it does not estimate the bias. 2.) For soil moisture, we found TC to work only for “anomalies” from the seasonal cycle. 3.) TC is sensitive to the climatology of the chosen reference data set. Triple Collocation

6. Surface soil moisture datasets: (converted to anomalies from the mean seasonal cycle) θ(A): ASCAT θ(L): AMSR-E LPRM θ(C):Catchment model Additive random error model (θ = truth, = 0) θ(A) = α(θ + (A)) θ(L) = λ(θ + (L)) θ(C) = γ(θ + (C)) α, λ, γ are calibration constants (rescale to account for systematic differences in variability between datasets) Stoffelen (1998), Scipal et al. (2008), Miralles et al. (2010)

7. Triple Collocation Set θ(A) as the reference dataset (α=1)  subscript “A” for estimated calibration constants and estimated errors. Solve for calibration: λ A ≈ / γ A ≈ / Solve for 2 = (RMSE TC ) 2 : A 2 (A) ≈ A 2 (L) ≈ A 2 (C) ≈ Assumption: Errors are not correlated with each other, nor with the true state.

8. RMSE(X) 2 = var(X) + var(true) – 2 R sqrt( var(X) var(true)) + bias 2 Interpretation of large-scale soil moisture RMSE “Because var true differs from place to place […] a single global RMSE target may not be appropriate.” (Entekhabi et al., 2010). At the global scale, we do not know var true (or even the mean):  Information on agreement is in time series correlation coefficient R.  Comparing soil moisture data sets using TC requires rescaling them to match the climatology of a reference data set.  RMSE TC estimates will depend on the climatology of the chosen reference data set.

9. Variability (anomalies) ASCAT AMSR-E/LPRM GEOS-5 RMSE of ASCAT (anom) [Ref dataset = ASCAT] [Ref dataset = AMSR-E] [Ref dataset = GEOS-5] Triple Collocation

10. RMSE and Fractional RMSE (fRMSE) Spatial differences in variability introduce nonlinearities: the ratio (and ranking!) of the domain-average RMSE TC in m 3 m -3 depends on the selected reference: Solution: Report errors as fractional RMSE: fRMSE = RMSE X (X) / var(X)

11. Variability (anomalies) ASCAT AMSR-E/LPRM GEOS-5 RMSE of ASCAT (anom) [Ref dataset = ASCAT] [Ref dataset = AMSR-E] [Ref dataset = GEOS-5] Fractional RMSE ASCAT AMSR-E/LPRM Draper et al. 2013, RSE, in press Estimated RMSE reflects variability of reference product. Use fractional RMSE instead. Triple Collocation

12. Uncertainty estimates indicate where fRMSE TC is reliable. ASCAT fRMSE TC AMSR-E fRMSE TC Width of 90% confidence interval for ASCAT fRMSE TC Width of 90% confidence interval for AMSR-E fRMSE TC Triple Collocation Draper et al. 2013, RSE, in press

13. fRMSE TC estimates can provide insights into relative skill. Consistent with expectations. Triple Collocation Draper et al. 2013, RSE, in press

14. fRMSE TC is reasonably insensitive to choice of data triplet. Residual differences between fRMSE TC estimates are related to error correlations between data products. Triple Collocation Assumptions Draper et al. 2013, RSE, in press In situ “errors” are not small! (scale mismatch, sensor issues,...) Assumption: Errors not correlated across datasets and with the truth. Check by adding a 4 th dataset (in situ).

15. Previous finding holds only if using anomalies from the mean seasonal cycle! Triple Collocation Assumptions

16. Error estimates for soil moisture retrievals Triple collocation Error propagation Soil moisture data assimilation Outline

17. Error propagation Propagate expected errors in input observations and parameters through the soil moisture retrieval model: ASCAT (Naeimi et al. 2007) LPRM (Parinussa et al. 2011) Used average of the error time series at each location. Assumed to represent errors in the soil moisture anomalies.

18. TC and EP yield reasonably consistent fRMSE patterns. ASCAT fRMSE TC AMSR-E fRMSE TC Triple Collocation (TC) and Error Propagation (EP) Draper et al. 2013, RSE, in press ASCAT fRMSE EP AMSR-E fRMSE EP

19. Triple Collocation fRMSE TC is consistent with: fRMSE from error propagation expectations (vegetation classes) Draper et al. 2013, RSE, in press

20. Error estimates for soil moisture retrievals Triple collocation Error propagation Soil moisture data assimilation Outline

21. Improvements from data assimilation Validated with in situ data Anomalies ≡ mean seasonal cycle removed Skill increases significantly through data assimilation. Similar improvements from AMSR-E/LPRM and ASCAT. Metric: Anom. time series corr. coeff. Draper et al. (2012), GRL, doi:10.1029/2011GL050655.

22. Improvements from data assimilation Skill increases significantly through data assimilation. Similar improvements from AMSR-E/LPRM and ASCAT. Draper et al. (2012), GRL, doi:10.1029/2011GL050655. Consistent with similar skill levels for the AMSR-E and ASCAT retrievals themselves…. …except for terrain with high topographic complexity (TC>10%).

23. Root zone soil moisture skill improvement from assimilation over model (ΔR) As long as (obs skill − model skill) > −0.2, assimilation improved the model skill. Draper et al. (2012), GRL, doi:10.1029/2011GL050655. Reichle et al. (2008), GRL, doi:10.1029/2007GL031986. Improvements from data assimilation Synthetic experiment

24. Both triple collocation and error propagation can estimate spatial patterns in the fRMSE. Good agreement with each other, and with expectations (based on vegetation, known problems) Triple collocation is believed to provide correct magnitude, and method is robust (dependent on careful selection of data sets, use of anomalies from the seasonal mean) Conclusions (1/4)

25. How should we evaluate remotely sensed soil moisture globally? Substantial spatial variation in RMSE and fRMSE across North America Evaluation based on handful of in situ sites may not represent global accuracy Triple collocation or error propagation could provide a useful complement It is unclear how current RMSE target accuracies can be interpreted globally Selection of reference determines magnitude of RMSE (and domain- average RMSE has non-linear dependence on reference) Uniform RMSE target over large domain not sensible Need alternative metrics (fRMSE, or see Crow et al. (2010), Entekhabi et al. (2010)) Conclusions (2/4)

26. Data assimilation: Assimilation of soil moisture retrievals improves model estimates of surface and root zone soil moisture. AMSR-E and ASCAT retrievals have comparable skill and yield similar improvements after data assimilation. Improvements can be realized even when the retrievals are (somewhat) less skillful than the model estimates. Conclusions (3/4)

27. How should we specify soil moisture observation error variances for data assimilation? Usually specified as a constant in the observation climatology. Uniform error variance not sensible, and resulting spatial patterns do not resemble other estimates. Use a uniform fRMSE, or spatial maps from triple collocation or error propagation (latter gives temporal variability). Conclusions (4/4)

28. Thank you for your attention. Questions? References: Draper, C. S., R. H. Reichle, R. de Jeu, V. Naeimi, R. Parinussa, and W. Wagner (2013), Estimating root mean square errors in remotely sensed soil moisture over continental scale domains, Remote Sensing of Environment, in press.Estimating root mean square errors in remotely sensed soil moisture over continental scale domains Draper, C. S., R. H. Reichle, G. J. M. De Lannoy, and Q. Liu (2012), Assimilation of passive and active microwave soil moisture retrievals, Geophysical Research Letters, 39, L04401, doi:10.1029/2011GL050655.Assimilation of passive and active microwave soil moisture retrievals

29. Back-Up Slides