Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 1 Chapter 13: Inferential Statistics Objectives Explain the concept of standard error and describe how sample size affects standard error. Explain how a test of significance is related to the null hypothesis and the research hypothesis of a study, and differentiate between one-tailed and two-tailed tests of significance. Define Type I and Type II errors.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 2 Chapter 13: Inferential Statistics Explain when and how to use a t test for independent samples, a t test for nonindependent samples, and gain or difference scores. Explain when and how to use analysis of variance, including one-way and multifactor ANOVA and ANCOVA. Explain when to use multiple regression and chi square.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 3 Concepts Underlying Inferential Statistics Inferential statistics are data analysis techniques for determining how likely it is that results obtained from a sample, or samples, are the same results that would be obtained from the entire population.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 4 Concepts Underlying Inferential Statistics Descriptive statistics show how often or how frequent an event or score occurred. Inferential statistics help researchers know whether they can generalize their findings to a population based upon their sample of participants. Inferential statistics use data to assess likelihood—or probability.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 5 Concepts Underlying Inferential Statistics Inferences about populations are based on information from samples. There is very little chance that any sample is identical to the population. The expected variance among sample means and the population mean is referred to as sampling error. Sampling error is expected. Sampling error tends to be normally distributed.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 6 Standard Error A distribution of sample means is normally distributed and has it’s own mean and standard deviation. The standard deviation of the sample means is referred to as the standard error of the mean. Our ability to estimate standard error of the mean is affected by size of sample. As the sample size increases the standard error of the mean decreases.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 7 Standard Error Our ability to estimate the standard error of the mean is also affected by the size of the population standard deviation. The standard error of the mean can be calculated by: SE x = the standard error of the mean SD = the standard deviation for a sample N = the sample size

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 8 Hypothesis Testing Hypothesis testing is the process of decision making in which researchers evaluate the results of a study against their original expectations. Research hypothesis: Predicting a difference in scores Null hypothesis: Predicting no difference in scores

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 9 Hypothesis Testing We want to assure differences we observe between groups are ‘real’ differences and did not occur by chance. If the groups are significantly different we reject the null hypothesis. We do not accept a research hypothesis, we can’t prove our hypothesis.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 10 Hypothesis Testing We instead report that our research hypothesis was supported. If there are not expected differences, we report that the null hypothesis was not rejected; and that our research hypothesis was not supported.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 11 Tests of Significance Tests of significance allow us to inferentially test if differences between scores in our sample are simply due to chance or if they are representative of the true state of affairs in the population.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 12 Tests of Significance To conduct a test of significance we determine a preselected probability level, known as level of significance (alpha or α). Usually educational researchers use alpha.05 or 5 out of 100 chances that the observed difference occurred by chance (α =.05).

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 13 Tests of Significance Two-tailed and one-tailed tests Tests of significance are almost always two- tailed. Researchers will select a one-tailed test of significance only when they are quite certain that a difference will occur in only one direction. It is ‘easier’ to obtain a significant effect when predicting in one direction.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 14 Type I and Type II Errors Based upon a test of significance the researcher will either reject or not reject the null hypothesis. The researcher makes a decision that the observed effect is or is not due to chance. This decision is based upon probability, not certainty.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 15 Type I and Type II Errors Sometimes the researcher will erroneously reject the null hypothesis or will erroneously retain the null hypothesis. When the researcher incorrectly rejects the null hypothesis she has committed a Type I error. When the researcher incorrectly fails to reject the null hypothesis but a true difference exists, she has committed a Type II error.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 16 Type I and Type II Errors True status of null hypothesis: True (Should not be rejected) True status of null hypothesis: False (Should be rejected) Researcher’s decision (True: does not reject) Correct DecisionType II Error Researcher’s decision (False: Rejects) Type I ErrorCorrect Decision

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 17 Degrees of Freedom After determining whether the significance test will be two-tailed or one-tailed and selecting a probability level (i.e., alpha), the researcher selects an appropriate statistical test to conduct the analysis. Degrees of freedom are the number of observations free to vary around a parameter. Each test of significance has its own formula for determining degrees of freedom (df ). The value for the df is important in determining whether the results are statistically significant.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 18 Tests of Significance The use of a specific significance test is determined by several factors. Scale of measurement represented by the data Participant selection Number of groups being compared Number of independent variables Significance tests applied incorrectly can lead to incorrect decisions.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 19 Tests of Significance The first decision in selecting an appropriate test is to determine whether a parametric or nonparametric test will be used. Parametric tests are generally more powerful and are generally preferred.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 20 Tests of Significance Parametric tests require that the data meet several assumptions. Variable must be normally distributed Interval or ratio scale of measurement Selection of participants is independent Variance of the comparison groups is equal Most parametric tests are fairly robust. If assumptions are violated, nonparametric tests should be used.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 21 Tests of Significance t test The t test is used to determine whether two groups of scores are significantly different from one another. The t test compares the observed difference between means with the difference expected by chance.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 22 Tests of Significance The t test for independent samples is a parametric test of significance used to determine if differences exist between the means of two independent samples. Independent samples are randomly formed. The assumption is that the means are the same at the outset of the study but there may be differences between the groups after treatment.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 23 Tests of Significance The t test for nonindependent samples is a parametric test of significance used to determine if differences exist between the means of two groups that are formed through matching. When scores are nonindependent, they are systematically related.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 24 Tests of Significance The comparison of gain or difference scores are not generally tested with a t- test. There are other better strategies for analyzing such data. e.g., t test on posttest scores (if there are no differences on pretest scores). e.g., Analysis of covariance (if there are differences in pretest scores).

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 25 Analysis of Variance A simple (one-way) analysis of variance (ANOVA) is a parametric test used to determine whether scores from two or more groups are significantly different at a selected probability level. ANOVA is used to avoid the error rate problems of conducting multiple tests.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 26 Analysis of Variance An F ratio is computed to determine if sample means are significantly different. The F ratio is calculated based upon variance between groups/variance within groups. The larger the F-ratio the more likely there are differences among groups.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 27 Analysis of Variance If there are significant differences among groups based upon an ANOVA; the researcher then must determine where these differences exist. Multiple comparisons are used to determine where differences between groups are statistically significant. Comparisons planned before collecting data are referred to as a priori. Comparisons after are referred to as post hoc.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 28 Analysis of Variance When a factorial design is used and there are two or more independent variables analyzed, a factorial or multifactor analysis of variance is used to analyze the data. MANOVA is an analytic procedure used when there is more than one dependent variable and multiple independent variables.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 29 Analysis of Variance ANCOVA is a form of ANOVA that allows for control of extraneous variables and also is used as a means for increasing power of an analysis. Power is increased in an ANCOVA because the within-group error variance is decreased. When a study has two or more dependent variables, and a covariate, MANCOVA is used.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 30 Multiple Regression Multiple regression is used to determine the amount of variance accounted for in a dependent variable by interval and ratio level independent variables. Multiple regression combines variables that are known to predict the criterion variable into an equation. Stepwise regression allows the researcher to enter one variable at a time.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 31 Multiple Regression Multiple regression is also the basis for path analysis. Path analysis begins with a predictive model. Path analysis determines the degree to which predictor variables interact with each other and contribute to variance in the dependent variables.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 32 Chi Square Chi Square is a nonparametric test used to test differences between groups when the data are frequency counts or percentages or proportions converted into frequencies. A true category is one in which persons naturally fall. An artificial category is one that is defined by the researcher.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 33 Other Statistical Procedures Data mining uses analytical tools to identify and predict patterns in datasets. Factor analysis is a statistical procedure used to identify relations among variables in a correlation matrix. Factor analysis is often used to reduce instruments to scales or subscales.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved. 34 Other Statistical Procedures Structural Equation Modeling (SEM) Structural equation modeling is similar to a combination of path analysis and factor analysis. SEM is a powerful analytic tool.