Copyright 2002 David M. Hassenzahl Using r and  2 Statistics for Risk Analysis

Copyright 2002 David M. Hassenzahl Objectives Purpose: to compare model to data –“validate model” (or not) Two techniques –r (correlation coefficient) –  2 (Chi-squared) Apply to a familiar problem (barium decay)

Copyright 2002 David M. Hassenzahl Statistics Descriptive Comparison –Z-scores, hypotheses –Confidence levels –Evaluating models –Correlation and Chi-squared

Copyright 2002 David M. Hassenzahl Confidence Levels Given 100 flips of a coin. Would you bet $1000 that the next flip will yield heads if –50 heads? –90 heads? –99 heads? –999 heads out of the last 1000 flips? How about for $5? For 50% of your current net worth?

Copyright 2002 David M. Hassenzahl Statistical Significance Z = –(sample occurrence – number in sample times expected probability –Divide by square root of (np(1-p) Student’s t One-sided versus two sided tests! “p values” Confidence intervals

Copyright 2002 David M. Hassenzahl Type I and II errors Type I: reject the truth! (accuracy) Type II: accept an untruth! (precision) This is important… there’s often a tradeoff here!

Copyright 2002 David M. Hassenzahl Z-scores Intuition Z score will be big if –Numerator: if xbar >>  OR  >> xbar –s is very small –n is very big Bigger Z-score: confidence that   xbar Small Z-score: confidence that   xbar

Copyright 2002 David M. Hassenzahl From Z’s to r’s and  2 r and  2 compare more than one estimate Compare –Set of model predictions to –Set of data or observations If r is SMALL (little correlation) the model doesn’t fit If  2 is SMALL then the model does fit

Copyright 2002 David M. Hassenzahl “Goodness of fit” We say that r and  2 evaluate “goodness of fit” Note that a good fit does not mean that the model is right!

Copyright 2002 David M. Hassenzahl Barium Decay Theory: barium is removed as a constant function of concentration “Exponential decay” C(T) = C(0)e kT –k = /min –C(0) = 0.16 mgBa / liter blood (From SWRI page 56 – 63; hypothetical)

Copyright 2002 David M. Hassenzahl Exponential Decay Model Figure 2-9 from Should We Risk It?

Copyright 2002 David M. Hassenzahl Sample blood at 1 hour intervals Time (hours)Measured Concentration

Copyright 2002 David M. Hassenzahl Measured and Expected Time (hours)Measured Concentration Predicted Concentration

Copyright 2002 David M. Hassenzahl Graphical Comparison After Figure 2-9 from Should We Risk It?

Copyright 2002 David M. Hassenzahl How well does the model fit? Why do we care? –Future predictions –Is there a better model? Looks OK. Is that good enough? Try our two tools: r and  2

Copyright 2002 David M. Hassenzahl r Conceptual Compares model predictions to the data Asks –“What if there is no relationship (or correlation) between model and data?” –Is the model as close to the average value of the x’s as it is to the actual x’s?

Copyright 2002 David M. Hassenzahl r terms or components Predicted mean and standard deviation Observed mean and standard deviation “Covariance” –Do they go up and down together? –If independent, covariance = 0 r = Covariance (predicted, observed) (STDEV O)  (STDEV P)

Copyright 2002 David M. Hassenzahl Means Observed x o bar = (  x oi ) /n x o bar = ( )/9 = Observed x p bar = (  x pi ) /n = 0.051

Copyright 2002 David M. Hassenzahl Standard Deviations

Copyright 2002 David M. Hassenzahl Covariance

Copyright 2002 David M. Hassenzahl Calculated r r = Covariance (predicted, observed) (STDEV O)  (STDEV P)

Copyright 2002 David M. Hassenzahl Intuition Behind r If there is no relationship between observed and predicted, r = 0 If r  0, positive correlation If r  0, negative correlation

Copyright 2002 David M. Hassenzahl r Discussed 0.99 seems reasonably good Is there a better fit What about theory? Limitations: even low correlations may be okay…just a screening tool

Copyright 2002 David M. Hassenzahl t test for r n-2 = 7 degrees of freedom Look it up in the Student’s-t table Accept model validity at 99% confidence level if Student’s t is greater than 2.998

Copyright 2002 David M. Hassenzahl Chi-squared This formula “normalizes” to the size of the individual x oi If all x oi  x ip,  2 = 0 Look up value in table (page 398)

Copyright 2002 David M. Hassenzahl Chi-squared 9 data points Suppose we are concerned with 99% confidence level We would need a chi-squared of greater than 21.7 to reject this line Calculating, we find that  2 = 0.06! Note that it still might be possible to find a better line, even with the exponential

Copyright 2002 David M. Hassenzahl Conclusion Both r and Chi-squared appear to validate this model Suggests that our theoretical idea about the model may be valid Doesn’t tell us we are right, just that we may be acceptably wrong!