McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. Educational Research: Fundamentals for the Consumer Woolfolk / Perry Child and Adolescent Development © 2012 Pearson Education, Inc. All rights reserved. Sixth Edition

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. Nonexperimental Quantitative Research Designs Chapter 7

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 3 Discussion Topics Nonexperimental research studies Descriptive studies Comparative studies Correlational studies Causal-comparative and Ex Post Facto designs Using surveys in quantitative studies

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 4 Nonexperimental Studies Research design - the plan and structure of research to provide a credible answer to a research question Purpose of nonexperimental designs Describe current existing characteristics such as achievement, attitudes, relationships, etc.

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 5 Nonexperimental Studies Four types of studies Descriptive Comparative Correlational Causal-comparative and ex post facto Survey

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 6 Descriptive Studies Studies that describe a phenomena Statistical nature of the description Frequency Percentages Averages Graphs Importance of these designs in the early stages of the investigation of an area

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 7 Descriptive Studies Criteria for evaluating descriptive studies Conclusions about relationships should not be drawn Participants and instruments should be described completely

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 8 Comparative Studies These studies investigate the relationship of one variable to another by examining differences on the dependent variable between two groups of participants If math scores for males are significantly higher than those for females, a relationship exists between gender and math achievement If the academic self-concept scores for ninth graders are significantly different than those for twelfth graders, a relationship exists between grade level and academic self-concept

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 9 Comparative Studies Criteria for evaluating these studies Participants and instruments are described completely Criteria for identifying the different groups is clearly stated No inferences are made about causation Graphs and images depict the results accurately

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 10 Correlational Studies Simple correlation studies Studies that examine the relationship between two variables Two variables Predictor and criterion Use caution describing the variables as independent and dependent Examples Math achievement and math attitudes Teacher effectiveness and teacher efficacy

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 11 Correlational Studies Simple correlation studies Cautions in interpreting correlations A relationship between two variables (e.g., achievement and attitude) does not mean one causes the other Possibility of low reliability of the instruments makes it difficult to identify relationships Lack of variability in scores (e.g., everyone scoring very, very low; everyone scoring very, very high; etc.) makes it difficult to identify relationships Large sample sizes and/or using many variables can identify significant relationships for statistical reasons, but they are not meaningful

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 12 Correlational Studies Prediction studies Designs that examine the predictive nature of the relationships between variables Two types of studies Simple prediction Multiple regression

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 13 Correlational Studies Prediction studies Simple predictive studies Performance on one variable (i.e., the predictor) is used to predict performance on a second variable (i.e., the outcome or criterion) Examples – Scholastic Aptitude Test (SAT) scores are used to predict college freshmen grade point averages – Scores from a mathematical attitude scale are used to predict math achievement scores Importance of the time interval between collecting the predictor and criterion variable data

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 14 Correlational Studies Prediction studies Simple predictive studies Factors influencing correlations – Possibility of low reliability of the instruments measuring the predictor and criterion variables makes it difficult to identify relationships – Length of time between the predictor and criterion variable data collection – Existence of many factors, not only the one being examined, that influence the criterion variable

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 15 Correlational Studies Multiple regression Studies that examine performance on several variables (i.e., predictor variables) to predict performance on a single outcome variable (i.e., criterion) Examples Scholastic Aptitude Test (SAT) scores, high school grade point average, and high school rank in class are used to predict college freshmen grade point average Math attitude scale scores, academic self-esteem scale scores, and prior math grades are used to predict math achievement scores

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 16 Correlational Studies Multiple regression Issues of concern Sample size of at least 10 subjects for each predictor variable Relationships among the predictor variables (i.e., colinearity)

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 17 Correlational Studies Logistic Regression Another type of multiple regression analysis used to examine the relationship between the predictor variables and dependent variable. The result is a prediction of whether the participant is a “case” or “non-case” Example: Overall GPA, gender, and special education classification used to determine if a student will pass or fail a standardized test.

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 18 Correlational Studies Significance of correlation coefficients Statistical significance Does a statistical relationship exist? Is the observed correlation significantly different from zero? Practical significance Does a relationship of practical importance exist? Coefficient of determination (r 2 ) - the percentage of the criterion variable variation that can be explained by the variation in the predictor variable

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 19 Correlational Studies Guidelines for interpreting the size of correlation coefficients Much larger correlations are needed for predictions with individual than with groups Crude group predictions can be made with correlations as low as.40 to.60 Predictions for individuals require correlations above.75 Exploratory studies Correlations of.25 to.40 indicate the need for further research Much higher correlations are needed to confirm or test hypotheses Multiple correlation coefficients of are common and usually indicate practical significance

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 20 Correlational Studies Criteria for evaluating correlational studies Causation should not be inferred from correlational studies The reported correlation should not be higher or lower than the actual correlation Practical significance should not be confused with statistical significance

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 21 Correlational Studies Criteria for evaluating correlational studies The size of the correlation should be sufficient for the use of the results Prediction studies should report the accuracy of predictions for new subjects Procedures for collecting data should be clearly indicated Correlation studies that claim explanation (causality) should be examined for alternative explanations.

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 22 Causal-Comparative Studies Causal-comparative studies Use of correlational models to investigate possible cause and effect relationships Sophisticated statistical models Path analysis Structural equation modeling Fundamental limitations of all correlational research designs apply

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 23 Causal-Comparative Studies Ex-post-facto studies Studies that investigate the relationships between independent and dependent variables in situations where it is impossible or unethical to manipulate the independent variable Example - what is the effect of pre-kindergarten (Pre- K) attendance on first grade achievement – Cannot mandate Pre-K attendance for children – Characteristics and resources of families who do and do not send their children to Pre-K may influence first grade achievement Similarities with correlational and experimental research studies

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 24 Causal-Comparative Studies Ex-post-facto studies Issues of concern Selecting participants who are as similar as possible on all characteristics except the independent variable Generalizing beyond the participants studied

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 25 Causal Comparative Studies Criteria for evaluating causal-comparative and ex post facto studies Primary purpose is to investigate causal relationships when experimental studies are not possible Presumed causal condition has already occurred Potential extraneous variables are considered Existing differences between groups being compared are controlled Causal conclusions are made with caution

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 26 Using Surveys A data collection method that is very useful in descriptive and correlational studies Versatile Efficient Generalizable Two types of survey designs Cross sectional designs Longitudinal designs

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 27 Using Surveys Cross sectional studies Information is collected from one or more groups at the same time Examples Student’s, teacher’s, administrator’s, and parent’s opinions regarding an extended school year Elementary, middle, and secondary teachers’ feelings toward a new school board policy Issue of concern - comparisons across groups can be the result of differences between participants within the groups Fifth and seventh graders opinions can be affected by a changes in the attendance zones of a school

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 28 Using Surveys Longitudinal studies - information is collected from the same participants over time Example - changes in the academic self- concept of students from the sixth to the twelfth grade Issues of concern Loss of subjects over time Difficulty tracking participants over time

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 29 Using Surveys Steps in designing a survey Define a purpose and objectives Develop the items – guidelines Use clear, unbiased, non-ambiguous language Keep it short and simple Use grammatically correct language Do not write leading items Use the same response scale for all items Be consistent with wording

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 30 Using Surveys Steps in designing a survey Pilot test representative participants Identify concerns – Clarity – Format – Responding – Directions – Time to complete

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 31 Using Surveys Internet-based surveys attachments Web pages Evaluations of online university survey research centersonline university survey research

McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. 32 Using Surveys Using internet-based surveys Advantages Reduced time and cost Easy access Quick responses Ease of creating data sets Disadvantages Limited to those with access to the technology Confidentiality and privacy issue See Table 7.2 for advantages and disadvantages of internet-based surveys