1. Stochastic Hydrology Hydrological Frequency Analysis (I) Fundamentals of HFA
Prof. Ke-Sheng Cheng
Department of Bioenvironmental Systems Engineering
National Taiwan University

2. General interpretation of hydrological frequency analysis
Hydrological frequency analysis is the work of determining the magnitude of hydrological variables that corresponds to a given exceedance probability. Frequency analysis can be conducted for many hydrological variables including floods, rainfalls, and droughts.
The work can be better perceived by treating the interested variable as a random variable.
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

3. Let X represent the hydrological (random) variable under investigation
Let X represent the hydrological (random) variable under investigation. A value xc associating to some event is chosen such that if X assumes a value exceeding xc the event is said to occur. Every time when a random experiment (or a trial) is conducted the event may or may not occur.
We are interested in the number of Bernoulli trials in which the first success occur. This can be described by the geometric distribution.
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

4. Geometric distribution
Geometric distribution represents the probability of obtaining the first success in x independent and identical Bernoulli trials.
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

5. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

6. Recurrence interval vs return period
Average number of trials to achieve the first success.
Recurrence interval vs return period
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

7. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

8. The general equation of frequency analysis
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

9. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

10. Collecting required data.
Estimating the mean, standard deviation and coefficient of skewness.
Determining appropriate distribution.
Calculating xT using the general eq.
It is apparent that calculation of involves determining the type of distribution for X and estimation of its mean and standard deviation. The former can be done by GOF tests and the latter is accomplished by parametric point estimation.
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

11. Data series for frequency analysis
Complete duration series
A complete duration series consists of all the observed data.
Partial duration series
A partial duration series is a series of data which are selected so that their magnitude is greater than a predefined base value. If the base value is selected so that the number of values in the series is equal to the number of years of the record, the series is called an “annual exceedance series”.
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

12. Extreme value series Data independency
An extreme value series is a data series that includes the largest or smallest values occurring in each of the equally-long time intervals of the record. If the time interval is taken as one year and the largest values are used, then we have an “annual maximum series”.
Data independency
Why is it important?
Annual exceedance series and annual maximum series are different.
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

13. Parameter estimation Method of moments Maximum likelihood method
Method of L-moments (Gaining more attention in recent years)
Depending on the distribution types, parameter estimation may involve estimation of the mean, standard deviation and/or coefficient of skewness.
Parameter estimation exemplified by the gamma distribution.
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

14. Gamma distribution parameter estimation
Gamma distribution is a special case of the Pearson type III distribution (with zero location parameter).
Gamma density
where , , and  are the mean, standard deviation, and coefficient of skewness of X (or Y), respectively, and  and  are respectively the scale and shape parameters of the gamma distribution.
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

15. MOM estimators
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

16. Maximum likelihood estimator
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

17. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

18. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

19. Evaluating bias of different estimators of coefficient of skewness
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

20. Evaluating mean square error of different estimators of coefficient of skewness
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

21. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

22. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

23. Techniques for goodness-of-fit test
A good reference for detailed discussion about GOF test is:
Goodness-of-fit Techniques. Edited by R.B. D’Agostino and M.A. Stephens, 1986.
Probability plotting
Chi-square test
Kolmogorov-Smirnov Test
Moment-ratios diagram method
L-moments based GOF tests
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

24. Probability plotting Fundamental concept
Probability papers
Empirical CDF vs theoretical CDF
Example of misuse of probability plotting
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

25. Suppose the true underlying distribution depends on a location parameter  and a scale parameter  (they need not to be the mean and standard deviation, respectively). The CDF of such a distribution can be written as
where Z is referred to as the standardized variable and G(z) is the CDF of Z.
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

26. where x represents the observed values of the random variable X.
Also let Fn(X) represents the empirical cumulative distribution function (ECDF) of X based on a random sample of size n. A probability plot is a plot of
on x
where x represents the observed values of the random variable X.
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

27. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

28. Most of the plotting position methods are empirical
Most of the plotting position methods are empirical. If n is the total number of values to be plotted and m is the rank of a value in a list ordered by descending magnitude, the exceedence probability of the mth largest value, xm, is , for large n, shown in the following table.
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

29. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

30. Misuse of probability plotting
Log Pearson Type III ?
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

31. Misuse of probability plotting
48-hr rainfall depth
Log Pearson Type III ?
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

32. How do we fit a probability distribution to a random sample?
Fitting a probability distribution to annual maximum series (Non-parametric GOF tests)
How do we fit a probability distribution to a random sample?
What type of distribution should be adopted?
What are the parameter values for the distribution?
How good is our fit?
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

33. Chi-square GOF test
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

34. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

35. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

36. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

37. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

38. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

39. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

40. Kolmogorov-Smirnov GOF test
The chi-square test compares the empirical histogram against the theoretical histogram.
In contrast, the K-S test compares the empirical cumulative distribution function (ECDF) against the theoretical CDF.
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

41. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

42. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

43. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

44. In order to measure the difference between Fn(X) and F(X), ECDF statistics based on the vertical distances between Fn(X) and F(X) have been proposed.
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

45. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

46. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

47. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

48. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

49. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

50. Hypothesis test using Dn
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

51. Values of for the Kolmogorov-Smirnov test
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

52. IDF curve fitting using the Horner’s equation
The intensity-duration-frequency (IDF) relationship of the design storm depths
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

53. DDF curves
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

54. IDF curves
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

55. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

56. Alternative IDF fitting (Return-period specific)
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

57. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

58. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

59. Further discussions on frequency analysis
Extracting annual maximum series
Probabilistic interpretation of the design total depth
Joint distribution of duration and total depth
Selection of the best-fit distribution
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

60. Annual maximum series
Data in an annual maximum series are considered IID and therefore form a random sample.
For a given design duration tr, we continuously move a window of size tr along the time axis and select the maximum total values within the window in each year.
Determination of the annual maximum rainfall is NOT based on the real storm duration; instead, a design duration which is artificially picked is used for this purpose.
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

61. Random sample for estimation of design storm depth
The design storm depth of a specified duration with return period T is the value of D(tr) with the probability of exceedance equals  /T.
Estimation of the design storm depth requires collecting a random sample of size n, i.e., {x1, x2, …, xn}.
A random sample is a collection of independently observed and identically distributed (IID) data.
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

62. Probabilistic interpretation of the design storm depth
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

63. It should also be noted that since the total depth in the depth-duration-frequency relationship only represents the total amount of rainfall of the design duration (not the real storm duration), the probability distributions in the preceding figure do not represent distributions of total depth of real storm events.
Or, more specifically, the preceding figure does not represent the bivariate distribution of duration and total depth of real storm events.
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

64. The usage of annual maximum series for rainfall frequency analysis is more of an intelligent and convenient engineering practice and the annual maximum data do not provide much information about the characteristics of the duration and total depth of real storm events.
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

65. Joint distribution of the total depth and duration
Total rainfall depth of a storm event varies with its storm duration. [A bivariate distribution for (D, tr).]
For a given storm duration tr, the total depth D(tr) is considered as a random variable and its magnitudes corresponding to specific exceedance probabilities are estimated. [Conditional distribution]
In general,
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

66. Selection of the best-fit distribution
Methods of model selection based on loss of information.
Akaike information criterion (AIC)
Schwarz's Bayesian information criterion (BIC)
Hannan-Quinn (HQIC) information criterion
Common practices of WRA-Taiwan
SE and U
SSE and SE
Can the p-value be used for selection of the best-fit distribution?
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

67. Information-criteria-based model selection
where is the log-likelihood function for the parameter  associated with the model, n is the sample size, and p is the dimension of the parametric space.
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

68. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

69. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

70. WRA Practice p: Number of distribution parameters
Weibull plotting position formula is used for calculation of cumulative probability.
12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU

71. 12/4/2018
Lab for Remote Sensing Hydrology and Spatial Modeling, Dept. of Bioenvironmental Systems Eng., NTU