Quantitative Research Lesson 1: Introduction

Quantitative Research Lesson Objectives After completing all activities in this lesson, learners will be able to… define research, explain the relationship of the scientific method to quantitative educational research, list the section headings of a quantitative research article and relate them to the scientific method, and locate answers to questions within a research article. Quantitative Research

What is research? Why do we need it? Research is answering a question with evidence (not the same as “library research”). We all have opinions about learning and about the most effective methods that should be used in education, but we need research as support for our opinions. Otherwise, they remain simply opinions. Quantitative Research

Quantitative Research What is research? Example: What is the best cell phone? There are lots of opinions. What kind of data would you choose to answer this question? Think for a minute and then go to the next slide. Quantitative Research

What is the best cell phone? If you are interested in… Features Safety Ease of use Cost Appearance You will probably want this type of data… Video/phone capabilities Battery charging safety Consumer Ratings Comparative prices Survey data on design, color, size, etc. Quantitative Research

Quantitative Research What is research? Conducting a study to answer a question. Example: What is the most effective way to teach reading? What type of data would you need to answer this question? Think for a minute and then go to the next slide. Quantitative Research

What is the best way to teach reading? You would probably want the following types of data comparing several methods: Test scores Oral reading samples Reading comprehension assessment results You would also need to consider various types of learners to see if the same method works for all of them. Quantitative Research

Research & the Scientific Method Research often follows the steps outlined in the “scientific method.” These steps are associated with quantitative research (data is in number form) instead of qualitative research (results are narrative in format). In quantitative research, you are trying to prove a theory (e.g. iPhones are better than Androids, would be a researcher’s theory) or there is no difference between an iPhone and an Android (would be a statistical hypothesis). Quantitative Research

Research & the Scientific Method Steps: Choose a general area of interest. Ask a question (or define a problem) See what others have done to answer the question. (the literature review section in an article is a summary of this information) State a hypothesis (your guess) about the answer. Collect and analyze data. Interpret results & draw conclusions – was your guess correct? Quantitative Research

Research & the Scientific Method Here’s a geology (specifically volcanoes) example: Choose a general area of interest. Volcanoes Ask a question (or define a problem) How can we predict volcanic eruptions? See what others have done to answer the question. Read research about the prediction of eruptions. Quantitative Research

Research & the Scientific Method Volcanoes example continued. 4. State a hypothesis (guess) about the answer. Swarms of small earthquakes come right before eruptions. 5. Collect and analyze data. Set up sensors around active volcanoes. 6. Interpret results & draw conclusions – was your guess correct? If we find a high correlation between earthquakes and eruptions, then our guess was correct. Quantitative Research

Research & the Scientific Method Here’s an education example: 1. Choose a general area of interest. Service learning (community service activities) 2. Ask a question (or define a problem) Do service learning experiences in middle school increase academic achievement? 3. See what others have done to answer the question. Read research about the effect of service learning on academic achievement. Quantitative Research

Scientific Method example cont’d. Educational example continued: 4. State a hypothesis (guess) about the answer. Middle school students who engage in service learning have higher test scores after their experiences than before. 5. Collect and analyze data. Standardized test scores before and after engaging in service learning. 6. Interpret results & draw conclusions – was your guess correct? If we find significantly higher scores after service learning, then our guess was (probably) correct. Quantitative Research

Format for Research Articles When you read a research article, the sections will generally follow the format of the scientific method. Take a moment to open and/or print this article that is in this week’s folder. We will be examining this article in detail as an example of educational research. (note the APA format) Martindale, T., Pearson, C., Curda, L, & Picher, J. (2005). Effects of an online instructional application on reading and mathematics standardized tests scores. Journal of Research on Technology in Education, 37(4), 349-360. Quantitative Research

Format for Research Articles Abstract (summary – convenient for the reader) Introduction (general problem area) Statement of the research problem or question Review of the literature (What have others done.) Statement of hypothesis (may not be present) Quantitative Research

Format for Research Articles (cont.) Methodology Subjects/Sample (These descriptions help Instruments the reader judge the validity* Procedures of the study and replicate the study.) Results (often includes statistics) Discussion (Was the “guess” correct?) References (articles referred to in the literature review) *note the new word, validity—we are no longer talking about credibility, which is used in qualitative research (more about validity later) Quantitative Research

Sample Research Article Read the abstract of the the Martindale, et al. article. Notice that it contains a short summary of the study, including results. In this article, the review of the literature comes right after the abstract. Notice that the literature review sets the stage by describing other similar studies and background on the factors involved. Quantitative Research

Sample Research Article cont’d The introduction (general problem area) provides a background for the use of standardized testing to evaluate the effectiveness of schools. In this article the literature review is part of the introduction. Quantitative Research

Sample Research Article cont’d The research question can be found in Purpose of the Study (p. 352): “The purpose of the current study was to determine if students who used FCAT Explorer scored higher on the FCAT reading and mathematics tests than those who did not use the software.” Quantitative Research

Sample Research Article cont’d The article does not state a specific hypothesis. The Methodology (Methods) section shows how the study was conducted. Notice that the sample (students who participated) is described in detail. Experimental Group refers to the students who used the software (the Control Group did not use the software). Quantitative Research

Sample Research Article cont’d The Results section contains tables of information and statistical results (like F=10.35, p<.01, f=.13). Statistical results will be explained later in the semester. For now, if the p value is less than .05, a statistically significant difference was found. Quantitative Research

Sample Research Article cont’d The Discussion and Conclusion sections explain what the numbers mean in answering the research question. Limitations are also noted in these sections. The authors’ positions and addresses/contact information are often listed next, depending on the journal format. Quantitative Research

Sample Research Article cont’d The References section contains a list of the articles used as citations in the article. They are presented using APA (American Psychological Association) format. Notice the difference in the way that books, book chapters (you will see “In”), articles, and web-based materials (you will see “Retrieved from”) are listed. NOTE: the College of Education has chosen APA as the format to be used in all programs, so you should be very familiar with it. Quantitative Research

Sample Research Article cont’d Read the sample research article and answer the questions. An answer will be available on the slide following each question. But, make sure you answer each question BEFORE reading the answer. Quantitative Research

Sample Research Article cont’d What does FCAT stand for? Check your answer on the next slide. Quantitative Research

Sample Research Article cont’d FCAT stands for the Florida Comprehensive Assessment Test. It appears to be similar to the old California system of mandated testing in various grades. Quantitative Research

Sample Research Article cont’d What is FCAT Explorer? Check your answer on the next slide. Quantitative Research

Sample Research Article cont’d FCAT Explorer is web-based test preparation software and resources for teachers, parents, and students. It was designed specifically to address the FCAT tests. Quantitative Research

Sample Research Article cont’d Describe the subjects. The subjects are the people who participate in the study. In qualitative research, they are most often called “participants” and in quantitative research, they are most often called “subjects.” How many schools and grades are included? Check your answer on the next slide. Quantitative Research

Sample Research Article cont’d A total of 3,961 students served as subjects in this study. See the top of page 353 for the breakdown by grade of the sample. See the next slide for more information about the sample. Quantitative Research

Sample Research Article cont’d Experimental Group 12 schools – 3 schools, Grade 4 Reading 3 schools, Grade 5 Math 3 schools, Grade 8 Math 3 schools, Grade 10 Math Chosen because of extensive use of FCAT Explorer Control Group 12 schools matched to each of the experimental schools Quantitative Research

Sample Research Article cont’d How were the Control Group schools matched with those in the Experimental Group? Check your answer on the next slide. Quantitative Research

Sample Research Article cont’d Schools were matched on district, size and performance data. Quantitative Research

Sample Research Article cont’d Why would matching be important? Check your answer on the next slide. Quantitative Research

Sample Research Article cont’d Matching is important because they are comparing test scores. It wouldn’t be fair to compare test scores of schools that were extremely different in previous years. If a school is already high and then uses the software, how would we know if the software made a difference, or if the school’s scores would have been high anyway? Quantitative Research

Sample Research Article cont’d What do the researchers mean by treatment? Check your answer on the next slide. Quantitative Research

Sample Research Article cont’d In this case, “treatment” means using the FCAT Explorer software. In most experimental research, whatever the experimental group does or is exposed to is called a treatment. The control group does not get the treatment. Later in the semester you will learn about comparison groups, who receive a different treatment rather than no treatment. Quantitative Research

Sample Research Article cont’d Here are some of the results (from Table 1) that were reported (in terms of test scores). Note: we use the term mean in research instead of average. The symbol is 5th grade Explorer use, ==284.48 5th grade Explorer non-use, =264.31 Does using FCAT Explorer cause higher test scores for 5th graders? Check your answer on the next slide. Quantitative Research

Sample Research Article cont’d Does using FCAT Explorer cause higher test scores for 5th graders? 5th grade Explorer use, =284.48 5th grade Explorer non-use, =264.31 You can see that 284.48 is higher than 264.31, but is that enough? Does that prove that using Explorer results in higher scores? See further explanations on the next slide. Quantitative Research

Sample Research Article cont’d By itself, a difference between two scores is not enough proof. There needs to be a statistically significant difference. Only statistical analyses can determine if there is a “statistically significant” difference. In this case, these results were reported (last paragraph on p. 353): F=4.46, p<.04, f=.09 Quantitative Research

Sample Research Article cont’d F=4.46, p<.04 (we’ll ignore the last f for right now) The F means that a particular statistical test (ANOVA-Analysis of Variance) was used. It computes statistical differences. The p is less than .05 (actually less than .04), so the difference is statistically significant. Later in the semester, you will learn what most of this means, but for now, remember that the “p” is the most important number. It is the probability that the results came about by chance, so p<.001 means that there is less than a .1% chance of that happening in Experiment 1 (so we’re 99.9% sure). When “p” is less than .05, the researchers find a statistically significant difference (or correlation in another type of research). The usual level of significance used in educational research is .05. Quantitative Research

Sample Research Article cont’d You may have noticed other statistics in the table. For example: SD=49.6 for 4th grade Explorer use. SD is “standard deviation” and indicates how wide a range of scores there were. It will be reported whenever a mean (remember that is the average) is shown. It is like saying “The average was 317.79, give or take 49.6.” The larger the SD, the wider the range of numbers. Quantitative Research

Sample Research Article cont’d Do you believe the results of this study? Why or why not? Check the next slide for possible answers. When you have finished, go to the next slide and try the Self-Check Exercise. Quantitative Research

Sample Research Article cont’d Pro: The researchers took a lot of care to match the schools for basic characteristics so the test scores, and not the school characteristics, were responsible for the differences. They also have a large number of subjects, which makes their study stronger and the results more believable. (Oh gosh, that is very different than qualitative research samples, isn’t it? Don’t forget.) Con: It is really hard to tell how much students actually used the FCAT Explorer program. Also, the program was more effective for younger students than older students. Why that happened needs to be addressed. Quantitative Research

GED 500 Quantitative Lesson 1 Self-Check Exercise GED 500 Quantitative Lesson 1 Quantitative Research

Quantitative Research Self-Check Exercise Which of the following statements is true? Research means looking up articles in ERIC or in journals in the library. Research means using the teaching method you like best. Research means answering a question with evidence. Quantitative Research

Quantitative Research Self-Check Exercise Which of the following statements is true? Research means looking up articles in ERIC or in journals in the library. Research means using the teaching method you like best. Research means answering a question with evidence. Quantitative Research

Quantitative Research Self-Check Exercise Which part of a research article matches this step in the scientific method? See what others have done to answer the question. Method/Methodology Literature Review Results Discussion Quantitative Research

Quantitative Research Self-Check Exercise Which part of a research article matches this step in the scientific method? See what others have done to answer the question. Method/Methodology Literature Review Results Discussion Quantitative Research

End of Lesson 1 Quantitative Research