1. Data Transformation and Uncertainty in Geostatistical Combination of Radar and Rain Gauges CSI 991 Dan Basinger

2. Abstract Quantitative Precipitation Estimates (QPE) can be performed using methods from geostatistics. Estimates are currently made by combining radar and rain gauge data. Problems arise, however, because of skewed and heteroscedastic nature of rainfall. Trans-Gaussian kriging techniques have been applied to correct for these problems.

3. Abstract (Contnued) Utilizing Kriging with External Drift (KED) in concert with the Box-Cox Power Transform has been applied to combined radar and rain gauge data.. Hourly precipitation data was collected in several places in Switzerland. KED measurements were performed with and without transformation.

4. Abstract (Continued) The effect of the transformation was examined in terms of accuracy of both point and probabilistic estimates. As a result, data transformation via kriging improved the analysis of the samples vis-à-vis the model Problems persist, however, in the case of dry gauges. In general, caution is emphasized in overutilizing the transformation due to positive bias.

5. Introduction Real-time radar-gauge combinations have been analyzed using different methodologies. – Interpolation with deterministic weights – Interpolation with splines – Objective analysis – Bayesian conditioning – For this study, a stochastic interpolation algorithm known as kriging is utilized.

6. Kriging for Geostatistical Analysis Variations of kriging have been used in the past for radar-rainfall gauge combinations. – Basic kriging – Kriging with external drift (which is utilized in this study) – Cokriging – Indicator kriging The choice of these methods can vary depending on the study scenario. Kriging with external drift (KED) has been selected for this study. The spatial covariance is derived from the data itself.

7. Kriging for Geostatistical Analysis (Continued) Kriging applied to radar-gauge combinations has resulted in improved point estimates. Probabilistic precipitation estimates have been generated in the form of a probability density function (PDF). – These probabilistic estimates drive the stochastic simulation of QPE estimates. – These in turn would find application in future hydrological forecasts.

8. Kriging for Geostatistical Analysis (Continued) The problem with geostatistical radar-gauge combinations is the fact that the underlying stochastic structure is based on the Gaussian distribution and the stationarity of: – Mean – Spatial covariance Rainfall is scattered in terms of wet and dry regions and thus, results in a skewed distribution. The Box-Cox Power Transform has been added to correct for problems related to this skewness.

9. Trans-Gaussian Kriging with the Box-Cox Transform Kriging with a power transform such as the Box-Cox method is called trans-Gaussian kriging. Similar applications include: – Mapping of hake distributions – Mapping of lead magnitudes in soils For radar rainfall gauge combinations throughout Switzerland, trans-Gaussian kriging is utilized to account for the non-Gaussian nature of the data. Kriging with external drift (KED) techniques are performed on the data and compared with results when no transform is applied. Hourly rainfall measurements from radar and rain gauges are utilized in this study.

10. Combination of Radar and Rainfall Gauges Analysis methods are based on the concept that precipitation data are perceived as a singular realization of a multivariate random variable. This variable is assumed to have a Gaussian distribution. – Mean also known as the drift of trend [deterministic] – Covariance function which describes the spatial dependence between the residuals [random] This distribution for precipitation generates a probabilistic estimate of the variable at any location.

11. Combination of Radar and Rainfall Gauges (Continued) The kriging process computes the expected value also known as the point estimate as well as the variance which is the prediction uncertainty within the distribution, Kriging can produce maps of the following: – Point estimates – Quartiles of the probabilistic estimates – Uncertainty estimates – Simulated realizations based on the observations

12. Combination of Radar and Rainfall Gauges (Continued) On this study, kriging with external drift (KED) has been selected. Mathematically, we have: Where Prec is precipitation with i,j being indices, alpha and beta are the trend parameters, the intercept and radar coefficient, and Z is a random process modeling the deviations of precipitation from the scaled radar field (Page 1334)

13. Combination of Radar and Rainfall Gauges (Continued) The data is collected at 75 gauge stations throughout Switzerland. Most of the time, a large fraction of these stations report dry conditions. The method of Restricted Maximum Likelihood (REML) is utilized to estimate trend and covariance parameters in the context of a relatively limited sample size.

14. Combination of Radar and Rainfall Gauges (Continued) Every data pair (radar and rainfall gauge) is examined individually. The radar- rainfall gauge matching is performed using the Nearest Neighbor Algorithm. – The radar value at a particular gauge location is derived from the radar pixel (1 km X 1 km). – This experiment is performed in such a way so as to decrease inconsistencies in spatial sampling between radar and rainfall gauges.

15. Application of Kriging with External Drift (KED) KED presupposes the random-process stochastic Z (i,j) to be Gaussian and stationary. – Constant Mean – Constant Variance – Constant Covariance As it turns out, the deviations from the radar- rainfall gauge relation are not Gaussian, but rather, positively skewed. – Precipitation values are truncated to 0 – Variance of the deviations is not constant, but increase with the precipitation values.

16. Application of Trans-Gaussian Kriging The algorithm utilized to correct for the deviations is trans-Gaussian kriging. – The method is used to transform the variable along with the covariates with a monotonic function. – The requirements for the Gaussian distribution are better fulfilled. Kriging is then performed on these transformed values. A back transformation is subsequently performed on the predictions resulting from the kriging process.

17. The Box-Cox Power Transform A useful set of transformations is the Box-Cox Power Transform. It is applicable to nonnegative data and has the format: Where Y and Y* are the original and transformed variables and λ is the transformation parameter. (Page 1334)

18. The Box-Cox Power Transform (Continued) Values of λ > 0 and λ < 1 are useful in correction for positive skewness. The case in which λ = 0 is not useful because of the preponderance of dry conditions. For positive values of λ, the minimum for Y* is -1/λ. For trans-Gaussian KED, this Box-Cox Power Transform is applied to both radar and rainfall gauge data. This is intended to correct for the skewness within both variables.

19. The Box-Cox Power Transform (Continued) The same value for λ is used for both sets of variables. Application of KED on the transformed data generates a probabilistic estimate at each location. – This yields a genuine Gaussian distribution with a mean value given by the point estimate. – This also yields a new variance corresponding to the kriging variance. The new Gaussian distribution is subsequently subjected to the Inverse Box-Cox Transform in order to obtain the probabilistic estimates from the original untransformed data.

20. The Box-Cox Power Transform (Continued) The point estimate for the precipitation values is approximated by the mean of the back- transformed quantiles. In the case of the lowest of the quantiles, the transformed values are set to -1/λ. Physically, these correspond to the dry precipitation cases. These quantiles are subsequently set to 0.

21. The Box-Cox Power Transform (Continued) The KED process is performed with four settings of L in the Box-Cox Transform. – If λ = 1, we have the classical application without transformation. – If λ = 0.5, we have the square root transformation which has been determined to be very suitable for radar-rainfall gauge combinations. – If λ = 0.1, we have an expression which is very close to a logarithmic transformation and not very suitable for radar-rainfall gauge combinations. – In a fourth case, we set λ = λ which is estimated case by case individually. e

22. The Box-Cox Power Transform (Continued) The fourth case deals with an estimate that turns out to work out very well to hourly precipitation values. There are several criteria that drive the selection of a value for case-dependent λ : – Gaussian distribution of residuals e – Relative linearity of the radar and rainfall gauge relationship.

23. The Box-Cox Power Transform (Continued) Applications of the Box-Cox transform over the course of a year have yielded values of 0.2 to 0.53 for λ. e Estimates of λ vary only slightly between seasons. e Values of λ vary more distinctly with rainfall variances e

24. The Box-Cox Power Transform (Continued) Large values of λ (> 0.3) usually occur in hours e of even and widespread rainfall. Small values of λ ( < 0.22) usually occur in e relatively limited and/or variable precipitation patterns. All general calculations are performed in “R” and the Box-Cox algorithm is performed in the “geoR” package.

25. Evaluation of the Methodology Several parameters are calculated in this study to access the quality of the point estimate. The ratio of total estimated to measured water amounts (RTW) measures the overall bias. The parameter called SCATTER is a measure of the error spread at wet locations. The Hansen-Kuipers Discriminant (HK) is utilized to measure the ability to differentiate between wet and dry areas.

26. Evaluation of the Methodology (Continued) The gauge measurements are compared with the probability density function applying the concept of a probability integral transform. We then calculate the frequency of observations that fall in between predefined quantiles. The Ratio of Observed to Expected frequencies based on interquantile bins measures the reliability of the kriging algorithm.

27. Evaluation of the Methodology (Continued) The Stable Equitable Error in Probability Space (SEEPS) is a triple category contingency value measuring the correspondence of dry, light, and intense rainfall conditions. Light rainfall is defined as the lower 67 % Intense rainfall is defined as the upper 33 %. The mean root transformed error is calculated as: n 2 MRTE= (1/n)Sum (SQRT(pred)–SQRT(obs) ) (Page 1336) i = 1 i i where “pred” refers to the point estimate “obs” refers to the gauge measurement.

28. Data Sources The source of the data for the trans-Gaussian radar and rainfall gauge measurements is hourly rainfall over the whole of Switzerland. A radar composite of hourly accumulated rainfall (in mm) covering the whole of Switzerland is placed on a grid space of 1 km X 1km. Rainfall gauge data is based on hourly accumulated rainfall with stations distributed evenly over the country.

29. Results Trans-Gaussian kriging has been utilized in the analysis of combined radar and rainfall gauge data. This was based on the hypothesis that data transformation can correct for the non-Gaussian nature of the data distributions. Kriging with external drift (KED) in conjunction with the Box-Cox Power Transform was utilized to modify the data distribution.

30. Results (Continued) Many improvements were observed when KED was utilized to generate a proper transformation as compared with the untransformed data. The underlying assumption of a Gaussian residual distribution along with relative stationarity of residual variance was better satisfied. Transformation changes the probabilistic estimate into a positively skewed probability distribution function. This properly scales the precipitation quantities correcting for unrealistic dependencies from KED without the Box-Cox Transform.

31. Results (Continued) KED without the Box-Cox Transform yielded rainfall distributions very similar to those from a parameter with exponents between 0.2 and 0.5. External transformations close to the logarithmic function (Box-Cox parameters less than 0.2) tended to produce a positive bias. Thus, in the overall analysis, it is important to avoid transformations close to the logarithmic. Other problems that the transformation has yet to rectify: – Cases with a high number of zeroes – Cases involving heteroscedasticity Cases in which the square root transform was derived produced good results.

32. Results (Continued) Overall, the best reliability for the probabilistic estimate was found in those case-dependent values of the transformation parameter λ. These resulted in optimization of Gaussian residuals. Case-by-case estimates can be generated on alternative criteria such as the linearity of the radar rainfall gauge relationship.

33. Conclusions Utilizing data transformations in the context of geostatistical applications is one way of analyzing the complex nature of rainfall data. Also there are currently limitations to this approach. These limitations can be potentially overcome by extending the trans-Gaussian kriging process by introducing additional terms such as: – Radar Quality – Topography – Precipitation Patterns and Structure

34. Selected References Box, G. E. P. and Cox, D.R. 1964: “An Analysis of Transformations” J. Roy. Stat. Soc. 26A: 211-252 Diggle, P.J. and Ribeiro, P.J. 2007 Model-Based Geostatistics ----------------- ----------------- Springer-Verlag Haberland, U. 2007 “Geostatistical Interpolation of Hourly Precipitation from Rain Gauges and Radar from a Large-Scale Extreme Rainfall Event” J. Hydrology 332: 144-157

35. Selected References (Continued) Hoel, P.G., Port, S.C., Stone, C.J. 1971 Introduction to Probability Theory Houghton Mifflin --------------------------------------------- Schabenberger, O. and Gotway, C.A. 2005 Statistical Methods for Spatial Data Analysis ---------------------------------------------------------- Chapman and Hall/CRC

36. Selected References (Continued) Schneider, S. and Steinacker, R. 2009 “Utilization of Radar Information to Refine Precipitation Fields by a Variational Approach” Journal of Meteor.Atmos. Physics 103 137-144 Sinclair, S. and Pegram, G. 2005 “Combining Radar and Rain Gauge Rainfall Estimates Using Conditional Merging” Atmospheric Science Letters 6: 19-22