For information contact H. C. Koons 30 October Preliminary Analysis of ABFM Data WSR 11 x 11-km Average Harry Koons 30 October 2003
For information contact H. C. Koons 30 October Scatter Plot of dBZ vs Emag WSR 11 x 11-km Average
For information contact H. C. Koons 30 October Approach Objective is to determine the probability of an extreme electric field intensity for a given radar return Use the statistics of extreme values to estimate the extreme electric field intensities –Reference: Statistical Analysis of Extreme Values, Second Edition, R. -D. Reiss and M. Thomas, Birkhäuser Verlag, Boston, 2001 As an example analyze WSR 11x11- km Average data Use both Peaks over Threshold (POT) and Maximum out of Blocks (MAX) methods Determine extreme value distribution functions for two-dBZ wide bins –For example the 0-dBZ bin is defined to be the range: 1 < dBZ < +1
For information contact H. C. Koons 30 October Extreme Value Methods Parametric models – n, n, n, n Parameters strictly hold only for the sample set analyzed Basic assumption is that the samples, x i, come from independent, identically distributed (iid) random variables –In our experience the techniques are very robust, as verified by the Q-Q plots, when this assumption is violated Analysis Methods –Peaks Over Threshold (POT) Take all samples that exceed a predetermined, high threshold, u. Exceedances over u fit by Generalized Pareto (GP) distribution functions –Maxima Out of Blocks (MAX) Take the maximum value within a pass, Anvil, etc. Tail of distribution is fit by extreme value (EV) distribution functions
For information contact H. C. Koons 30 October Generalized Pareto (GP) Distribution Functions (Peaks-over-Threshold Method) Exponential (GP0, = 0): Pareto(GP1, > 0): Beta (GP2, < 0): The – Parameterization:
For information contact H. C. Koons 30 October Standard EV Distribution Functions (Maximum out of Blocks Method) Gumbel (EV0, = 0): Fréchet (EV1, > 0): Weibull (EV2, < 0): The – Parameterization:
For information contact H. C. Koons 30 October Tutorial Example Based on a Manufactured Gaussian (Normal) Distribution Function Select 100 random samples uniformly from a normal distribution –Mean value, = 5 –Standard deviation, = 1 Maximum Likelihood Estimates for a normal model are the sample mean and sample standard deviation – = – = In general you maximize the likelihood function by taking appropriate partial derivates
For information contact H. C. Koons 30 October Distribution function, F, of a real- valued random variable X is given by F (X) = P{ X x } Histogram The density function, f, is the derivative of the distribution function –Sample Density –Model Density –Kernel Density Distribution and Density Functions
For information contact H. C. Koons 30 October Quantile Functions and the Q-Q Plot
For information contact H. C. Koons 30 October T-Year Values T-year level is a higher quantile of the distribution function T-year threshold, u(T), is the threshold such that the mean first exceedance time is T years –u(T) = F -1 (1-1/T) –1-1/T quantile of the df The T-year threshold also is exceeded by the observation in a given year with the probability of 1/T
For information contact H. C. Koons 30 October Scatter Plot of dBZ vs Emag WSR 11 x 11-km Average; 0 dBZ bin
For information contact H. C. Koons 30 October Cumulative Frequency Curve for Emag –1 < dBZ < +1
For information contact H. C. Koons 30 October Sample Statistics -1 < dBZ < +1 Sample Size, N = 271 Minimum = 0.02 kV/m Maximum = 2.41 kV/m Median = 0.6 kV/m Mean = 0.67 kV/m Choose Peaks Over Threshold (POT) Method for Extreme Value Analysis u = 1.0 kV/m (high threshold) n = 57 (samples over threshold) k = 32 (mean value of the stability zone)
For information contact H. C. Koons 30 October Gamma Diagram Point of stability is on The plateau between 30 and 40
For information contact H. C. Koons 30 October Kernel Density Function and Model Density Function Peaks Over Threshold Method (POT) –N = 271 samples between –1 and +1 dBZ –u = 1.0 kV/m (high threshold) –n = 57 (samples over threshold) MLE (GP0) – k = 32 (mean zone of stability) – = 0.0 – = (~left end point) – =
For information contact H. C. Koons 30 October Sample and Model Distribution Functions
For information contact H. C. Koons 30 October Sample and Model Quantile Functions
For information contact H. C. Koons 30 October Q – Q Plot Sample Model
For information contact H. C. Koons 30 October T-Sample Electric Field Intensity -1 < dBZ < +1 T, SamplesProbability, p Quantile, p kV/m , , , ,000, ,000,
For information contact H. C. Koons 30 October Extend Analysis to Other Bins Use bins centered at –2, 0, +2, and +4 dBZ Kernel Density Plot Sample Statistics Model Parameters T-Sample Electric Field Intensity
For information contact H. C. Koons 30 October Sample Kernel Densities for Exceedances Blk: -2 dBZ Red: 0 dBZ Grn: +2 dBZ Blu: +4 dBZ Emag kV/m
For information contact H. C. Koons 30 October Sample Statistics and Model Parameters -2 dBZ0 dBZ+2 dBZ+4 dBZ u, kV/m N k 0.0
For information contact H. C. Koons 30 October T-Sample Electric Field Intensity, kV/m T-Sample-2 dBZ0 dBZ+2 dBZ+4 dBZ , , , ,000, ,000,
For information contact H. C. Koons 30 October Maximum Out of Blocks Method #1 One Block is One Pass Analyze the 0 dBZ bin 38 Blocks (unique pass numbers) Maximum Emag for each block shows a slight dependence on sample-count per block –We will ignore this MLE (EV) Model Parameters = = = 0.322
For information contact H. C. Koons 30 October Sample Count vs. Pass Number Max for Passes
For information contact H. C. Koons 30 October Number of Occurrences vs. Sample Count Max for Passes
For information contact H. C. Koons 30 October Plot of Maximum Emag vs. Sample Count MAX for Passes
For information contact H. C. Koons 30 October Density Functions MAX for Passes
For information contact H. C. Koons 30 October Q-Q Plot MAX for Passes
For information contact H. C. Koons 30 October T-Pass Electric Field Intensity -1 < dBZ < +1 TEmag, kV/m , , , ,
For information contact H. C. Koons 30 October Maximum Out of Blocks Method #2 One Block is One Anvil Analyze the 0 dBZ bin 21 Blocks (unique anvil numbers) MLE (EV) Model Parameters = = = Right End Point = 10.5 kV/m
For information contact H. C. Koons 30 October Density Functions MAX for Anvils
For information contact H. C. Koons 30 October Distribution Functions MAX for Anvils
For information contact H. C. Koons 30 October Q-Q Plot MAX for Anvils
For information contact H. C. Koons 30 October T-Anvil Electric Field Intensity -1 < dBZ < +1 TEmag, kV/m , , , ,