1. 1 Chapter 15 Data Analysis: Basic Questions © 2005 Thomson/South-Western

2. 2 Choosing a Statistical Test Number of Variables Univariate Analysis Multivariate Analysis One Two or More

3. 3 Figure 1: Chart for Choosing Among Statistical Tests Level Meas Predictor(s)? N/O: t-test or z-test, ANOVA I/R: Regression, Conjoint Level of Measurement? N: Cluster analysis O: Kendall’s Coef of Concordance I: Factor analysis, Cluster analysis, MDS Interdependence Level of Measurement? N/O: Cross-tabs Chi-square I/R: Correlation Prediction #Dep Vars? 1 0 Level Meas Dep Var? Level Meas Predictor(s)? N: Contingency Coef, Index Pred. Assoc, Logit, z-test proportion O: Spearman Rank Corr (w ordinal dep var) I: Discriminant Analysis I/R N/O Level of Measurement? N: Frequencies, %’s, Chi-square O: Kolmogorov-Smirnov I/R: means Total# Vars? 2 1 3+

4. 4 Source: Appendix 15 Quick Stats Review

5. 5 Research Realities: Typical Hypothesis Testing Procedure 1)Specify Null and Alternative Hypotheses after analyzing the research problem 2) Choose appropriate statistical test, considering research design, and sampling distribution for the test statistic 3) Specify the significance level (alpha) for the problem being investigated 4) Collect the data, compute the value of the test statistic appropriate for the sampling distribution 5) Determine probability of the test statistic under the null Hypothesis using the sampling distribution in step 2 6) Compare the obtained probability with the specified significance level, reject or do not reject the null hypothesis on the basis of the comparison

6. 6 Judicial Analogy Illustrating Decision Errors True Situation: Defendent Verdict Innocent Guilty Innocent Guilty Error Probability:  Correct Decision Probability: 1-  Error Probability:  Correct Decision Probability: 1- 

7. 7 Types of Errors in Hypothesis Testing True Situation: Null Hypothesis Research Conclusions True False Reject H 0 Error: Type I Significance Level Probability:  Correct Decision Confidence Level Probability: 1-  Error: Type II Probability:  Correct Decision Power of Test Probability: 1-  Do Not Reject H 0

8. 8 Figure 1: Probability of z = 1.50 with a one-tailed test z=1.50 Area covered by arrow = 0.9332

9. 9 Figure 2: Computation of β error and power for several assumed true population proportions for the hypothesis, π<.20 Panel A: Critical Proportion under Null Hypothesis Panel B: Probability of realizing critical proportion when π =.22, which means null hypothesis is false. Panel C: Probability of realizing critical proportion when π =.21, which means null hypothesis is false. Panel D: Probability of realizing critical proportion when π =.25, which means null hypothesis is false. π =.20 π =.22 π =.21 π =.25 area =.05 p=.2263 Probability = β error =.648 Power =.352 Probability = β error =.8413 Power =.1587 Probability = β error =.0856 Power =.9144

10. 10 Figure 3: Power Function for Data in Table 3 Power = 1-β Probability of Rejecting Ho: π <.20 A F E C D B KEY: A: Type 1 error; true null hypothesis is rejected; significance level B: Type I error; true null hypothesis is rejected C: No error, false null hypothesis is rejected D: No error, true null hypothesis is not rejected E: No error, true null hypothesis is not rejected, confidence level F: Type II error; fall null hypothesis is not rejected

11. 11 Error and Power for Different Assumed True Values of  and the Hypothesis H 0 : .20.20.21.22.23.24.25.26.27.28.29.30 (.950)=1- .8413.6480.4133.2133.0856.0273.0069.0014.0005.0000 (.05)= .1587.3520.5867.7867.9144.9727.9931.9986.9995 1.0000 Value of  Probability of Type II or  Error Power of the Test 1- 

12. 12 Common Misinterpretations of What “Statistically Significant” Means *Viewing p values as if they represent the probability that the results occurred because of sampling error, e.g., p=.05 implies there is only a.05 probability that the results were caused by chance. *Equating statistical significance with practical significance. *Viewing the  or p levels as in some way related to the probability that the research hypothesis captured in the alternative hypothesis is true, e.g., a p value such as p<.001 is “highly significant” and therefore more valid than p<.05.