1. Chapter 14 Inferential Data Analysis

2. Inferential Statistics
Techniques that allow us to study samples and then make generalizations about the population. Inferential statistics are a very crucial part of scientific research in that these techniques are used to test hypotheses

3. Uses for Inferential Statistics
Statistics for determining differences between experimental and control groups in experimental research
Statistics used in descriptive research when comparisons are made between different groups
These statistics enable the researcher to evaluate the effects of an independent variable on a dependent variable

4. Sampling Error
Differences between a sample statistic and a population parameter because the sample is not perfectly representative of the population

5. Hypothesis Testing
The purpose of the statistical test is to evaluate the null hypothesis (H0) at a specified level of significance (e.g., p < .05)
In other words, do the treatment effects differ significantly so that these differences would be attributable to chance occurrence less than 5 times in 100?

6. Hypothesis Testing Procedures
State the hypothesis (H0)
Select the probability level (alpha)
Determine the value needed for significance
Calculate the test statistic
Accept or reject H0

7. Statistical Significance
A statement in the research literature that the statistical test was significant indicates that the value of the calculated statistic warranted rejection of the null hypothesis
For a difference question, this suggests a real difference and not one due to sampling error

8. Parametric Statistics
Techniques which require basic assumptions about the data, for example:
normality of distribution
homogeneity of variance
requirement of interval or ratio data
Most prevalent in HHP
Many statistical techniques are considered robust to violations of the assumptions, meaning that the outcome of the statistical test should still be considered valid

9. t-tests Characteristics of t-tests
requires interval or ratio level scores
used to compare two mean scores
easy to compute
pretty good small sample statistic

10. Types of t-test One-Group t-test Independent Groups t-test
t-test between a sample and population mean
Independent Groups t-test
compares mean scores on two independent samples
Dependent Groups (Correlated) t-test
compares two mean scores from a repeated measures or matched pairs design
most common situation is for comparison of pretest with posttest scores from the same sample

11. Hypothesis Testing Errors
Hypothesis testing decisions are made without direct knowledge of the true circumstance in the population. As a result, the researcher’s decision may or may not be correct
Type I Error
Type II Error

12. Type I Error
. . . is made when the researcher rejects the null hypothesis when in fact the null hypothesis is true
probability of committing Type I error is equal to the significance (alpha) level set by the researcher
thus, the smaller the alpha level the lower the chance of committing a Type I error

13. Type II Error
. . . occurs when the researcher accepts the null hypothesis, when in fact it should have been rejected
probability is equal to beta (B) which is influenced by several factors
inversely related to alpha level
increasing sample size will reduce B
Statistical Power – the probability of rejecting a false null hypothesis
Power = 1 – beta
Decreasing probability of making a Type II error increases statistical power

14. Hypothesis Truth Table
NULL HYPOTHESIS
TRUE
FALSE
CORRECT
DECISION
TYPE II
ERROR
ACCEPT
DECISION
TYPE I
ERROR
CORRECT
DECISION
REJECT

15. ANOVA - Analysis of Variance
A commonly used family of statistical tests that may be considered a logical extension of the t-test
requires interval or ratio level scores
used for comparing 2 or more mean scores
maintains designated alpha level as compared to experimentwise inflation of alpha level with multiple t-tests
may also test more than 1 independent variable as well as interaction effect

16. One-way ANOVA
Extension of independent groups t-test, but may be used for evaluating differences among 2 or more groups

17. Repeated Measures ANOVA
Extension of dependent groups t-test, where each subject is measured on 2 or more occasions
a.k.a “within subjects design”
Test of sphericity assumption is recommended

18. Random Blocks ANOVA
This is an extension of the matched pairs t-test when there are three or more groups or the same as the matched pairs t-test when there are two groups
Participants similar in terms of a variable are placed together in a block and then randomly assigned to treatment groups

19. Factorial ANOVA
This is an extension of the one-way ANOVA for testing the effects of 2 or more independent variables as well as interaction effects
Two-way ANOVA (e.g., 3 X 2 ANOVA)
Three-way ANOVA (e.g., 3 X 3 X 2 ANOVA)

20. Assumptions of Statistical Tests
Parametric tests are based on a variety of assumptions, such as
Interval or ratio level scores
Random sampling of participants
Scores are normally distributed
N = 30 considered minimum by some
Homogeneity of variance
Groups are independent of each other
Others
Researchers should try to satisfy assumptions underlying the statistical test being used

21. Improving the Probability of Meeting Assumptions
Utilize a sample that is truly representative of the population of interest
Utilize large sample sizes
Utilize comparison groups that have about the same number of participants

22. Two-Group Comparison Tests
a.k.a. Multiple Comparison or Post Hoc Tests
The various ANOVA tests are often referred to as “omnibus” tests because they are used to determine if the means are different but they do not specify the location of the difference
if the null hypothesis is rejected, meaning that there is a difference among the mean scores, then the researcher needs to perform additional tests in order to determine which means (groups) are actually different

23. Common Post Hoc Tests
Multiple comparison (post hoc) tests are used to make specific comparisons following a significant finding from ANOVA in order to determine the location of the difference
Duncan
Tukey
Bonferroni
Scheffe
Note that post hoc tests are only necessary if there are more than two levels of the independent variable

24. Analysis of Covariance
ANOVA
ANOVA design which statistically adjusts the difference among group means to allow for the fact that the groups differ on some other variable
frequently used to adjust for inequality of groups at the start of a research study

25. Nonparametric Statistics
Considered assumption free statistics
Appropriate for nominal and ordinal data or in situations where very small sample sizes (n < 10) would probably not yield a normal distribution of scores
Less statistical power than parametric statistics

26. Chi Square
A nonparametric test used with nominally scaled data which are common with survey research
The statistic is used when the researcher is interested in the number of responses, objects, or people that fall in two or more categories

27. Single Sample Chi-Square
a.k.a one-way chi-square or goodness of fit chi-square
Used to test the hypothesis that the collected data (observed scores) fits an expected distribution
i.e. are the observed frequencies and expected frequencies for a questionnaire item in agreement with each other?

28. Independent Groups Chi-Square
a.k.a. two-way chi-square or contingency table chi-square
Used to test if there is a significant relationship (association) between two nominally scaled variables
In this test we are comparing two or more patterns of frequencies to see if they are independent from each other

29. Overview of Multivariate Tests
Univariate statistic –
used in situations where each participant contributed one score to the data analysis, or in the case of a repeated measures design, one score per cell
Multivariate statistic –
used in situations where each participant contributes multiple scores

30. Example Multivariate Tests
MANOVA
Canonical correlation
Discriminant analysis
Factor analysis

31. Multiple Analysis of Variance
MANOVA
Analogous to ANOVA except that there are multiple dependent variables
Represents a type of multivariate test

32. Prediction and Regression Analysis
Correlational technique
Simple prediction
Predicting an unknown score (Y) based on a single predictor variable (X)
Y’ = bX + c
Multiple prediction
Involves more than one predictor variable
Y’ = b1X1 + b2X2 + c

33. Multiple Regression/Prediction
a.k.a multiple correlation
Determines the relationship between one dependent variable and 2 or more predictor variables
Used to predict performance on one variable from another
Y’ = b1X1 + b2X2 + c
Standard error of prediction is an index of accuracy of the prediction

34. Statistical Power
The probability that the statistical test will correctly reject a false null hypothesis
. . . it is effectively the probability of finding significance, that the experimental treatment actually does have an effect
a researcher would like to have a high level of power

35. Statistical Power alpha = probability of a Type I error
rejecting a true null hypothesis
this is your significance level
beta = probability of a Type II error
failing to reject a false null hypothesis
Statistical power = 1 - beta

36. Factors Affecting Power
Alpha level
Sample size
Effect size
One-tailed or two-tailed test

37. Alpha level
Reducing the alpha level (moving from .05 to .01) will reduce the power of a statistical test. This makes it harder to reject the null hypothesis

38. Sample size
In general, the larger the sample size the greater the power. This is because the standard error of the mean decreases as the sample size increases

39. One-tailed versus two-tailed tests
It is easier to reject the null hypothesis using a one-tailed test than a two-tailed test because the critical region is larger

40. Effect size
This is an indication of the size of the treatment effect, its meaningfulness
With a large effect size, it will be easy to detect differences and statistical power will be high
But, if the treatment effect is small, it will be difficult to detect differences and power will be low

41. Effect Size
Numerous authors have indicated the need to estimate the magnitude of differences between groups as well as to report the significance of the effects
One way to describe the strength of a treatment effect, or meaningfulness of the findings, is the computation of “effect size” (ES)
ES =
M1 - M2 SD
Note: SD represents the standard deviation of the control group or the pooled standard deviation if there is no control group

42. Effect Size Interpretation of ES by Cohen (1988)
0.2 represents a small ES
0.5 represents a moderate ES
0.8 represents a large ES
Researchers using experimental designs are advised to provide post hoc estimates of ES for any significant findings as a way to evaluate the meaningfulness

43. A Priori Procedures
Calculate the power for each of the statistical procedures to be applied
requires three indices - alpha, sample size, effect size
Estimate the sample size needed to detect a certain effect (ES) given a specific alpha and power
may require an estimation of ES from previous published studies or from a pilot study