Educational Research Chapter 13 Post-Analysis Considerations Gay, Mills, and Airasian 8th Edition (Electronic Reserves)

Topics Discussed in this Chapter Verification and storage of data Checking, organizing, and evaluating data Interpretation of results Hypothesized and unhypothesized results Statistical issues Methodological issues Statistical and practical significance Replication

Verifying and Storing Data Three issues Checking data Organizing data Evaluating the research conclusions Checking data for accuracy Check the original data Check the coded data against the initial uncoded data Electronic data Print the data sets and check them Check the output carefully Objective 1.1

Verifying and Storing Data Organizing data Label the data Dates Important details related to the data itself and the study Store the data Multiple copies Multiple sites Objective 1.2

Verifying and Storing Data Evaluate the research conclusion(s) Check the accuracy of the computations Check the reasonableness of the results Objective 1.3

Interpreting the Results Interpretations are made in light of … the purpose of the study, the original research hypotheses, and the findings of other related studies. Hypothesized results Results are interpreted relative to their support or lack of support of the research hypotheses Objectives 2.1 & 3.1

Interpreting Results Hypothesized results (continued) Supported hypotheses Probabilistic nature of the decision Easily interpreted given the control and manipulation used in the research design Non-supported hypotheses Possible importance of knowing what doesn’t work Potential methodological problems Objectives 2.1 & 3.1

Interpreting Results Unhypothesized results Results that appear during the study Must be interpreted with great care Researchers base their studies on ideas related to what will happen, not what is happening Possible importance to and influence on later studies Objective 4.1 & 4.2

Interpreting Results Statistical influences The sample must be representative of the population The assumptions of the specific statistical tests of significance must be met Objective 5.1

Interpreting Results Three methodological influences Measurement error – large amounts of error hamper the ability to find statistically significant results Statistical power The probability of avoiding a Type II error (failure to reject a false null hypothesis [something is there but you say there isn’t]) Objectives 5.2 & 6.1

Interpreting Results Three methodological factors (cont.) Statistical power (cont.) Three influential factors Sample size – the larger the sample the greater the power Significance levels Increasing alpha (e.g., .01, .05, .10) increases power If you say that there is a 5% chance that the differences are due to chance (null) then you are more likely to accept a false null hypothesis than if you set the value at a 10% chance. Objectives 5.2 & 6.2

Interpreting Results Three methodological factors (cont.) Statistical power (cont.) Effect size Larger effect sizes increase power Effect sizes can be estimated from prior studies, estimated by minimal levels below which the results are deemed unimportant, and agreed upon levels of small (.20), moderate (.50), and large (.80). Objectives 5.2 & 6.2

Interpreting Results Statistical and practical significance Statistical significance relates to the tests of significance being used Practical significance relates to the usefulness of the results Statistical and practical significance are not necessarily the same Statistical significance depends on sample size Studies with large samples can be significant but of little value Objective 5.3

Interpreting Results Replication Replication involves repeating the study Different subjects Different context Different times Extremely important when… an unusual or new result is found, or the results have high levels of practical significance. Objective 7.1