More studies are better than less studies. Let me ask a question. meta.ppt

Dave told us that some research studies are more powerful than others. I was wondering if there is any procedure in education that examines collections of studies? Northwest CLD meta.ppt

I am glad you asked that question. There is a procedure for ‘aggregating*’ data from a variety of studies. It is called meta- analysis. *Aggregate: To gather into a mass or whole. meta.ppt

In meta-analysis, a researcher examines many studies on a particular strategy and derives a numerical indicator of the relative effectiveness of the strategy, averaged across all studies. This indicator is called an effect size (ES). Such an analysis of research not only provides a numerical indicator of the relative effect of a particular intervention, but the effect size also allows comparison with other approaches used in special education or related services. Thanks NWCLD presenters. We ARE getting a lot of great evidence based practices from a research-summary procedure called meta-analysis. Check out what Steve Forness and his buddies wrote in Teaching Exceptional Children… 1 2 I heard something about this ‘ES’ in another special education class. Could you review how this works. 3 meta.ppt

I’d love to. Let’s begin with what you know about the ‘normal bell-shaped curve since it is at the heart of summarizing studies. First, what does the curve look like? I’d love to. Let’s begin with what you know about the ‘normal bell-shaped curve since it is at the heart of summarizing studies. First, what does the curve look like? meta.ppt

I hope I get the right answer: “It looks like a bell.” meta.ppt

Let’s talk about a normal distribution – the bell-shaped curve. The peak of the distribution, which occurs at the ‘mean’ (the average of all the numbers).. Cite examples. Math performance, IQ, shoe size. meta.ppt

Each side of the curve has inflection points where the curve makes shifts in direction. meta.ppt

The first inflection point to the right of the mean occurs one ‘standard deviation’ (SD) above the mean. Mean + 1 SD 1 SD Above Below meta.ppt

The second inflection point to the right of the mean occurs two standard deviations above the mean. Mean + 1 SD Mean + 2 SD Above Below meta.ppt

Inflection points below the mean also occur at one and two standard deviations below the mean. Mean + 1 SD Mean + 2 SD Mean - 1 SD Mean - 2 SD Above Below meta.ppt

Mean +1 SD +2 SD SD -2 SD Because all points are in reference to the mean, we will indicate the differences with the mean as plus or minus standard deviations.. We say, “+1 Standard Deviation” We say, “-2 Standard Deviations” Above Below meta.ppt

Mean +1 SD +2 SD SD -2 SD One of the most useful aspects of the normal curve is that there are specific areas under each part of the curve. meta.ppt

Mean +1 SD +2 SD SD -2 SD As we can see, 50% of the data (Math scores, IQs, shoe sizes) fall on each side of the mean. 50% meta.ppt

Mean +1 SD +2 SD SD -2 SD Of this 50%, 34% falls between the mean and one standard deviation above and one standard deviation below the mean. 50% 34% Above Below meta.ppt

Mean +1 SD +2 SD SD -2 SD The area between one and two standard deviations from the mean holds 14% of the distribution. 50% 34% 14% Above Below meta.ppt

Mean +1 SD +2 SD SD -2 SD Since the total area on each side must total 50%, we know there is 2% of the distribution beyond two standard deviations (SD) in each direction. 50% 34% 14% 2% Above Below meta.ppt

Mean +1 SD +2 SD SD -2 SD Generally, the ‘average range’ is considered between -1 and +1 standard deviations (SD). 50% 34% 14% 2% Average Range meta.ppt

Here I am back at Diamondhead. Now that you can see characteristics of the bell-shaped curve, please take out the copy you received earlier. How about now. Let’s mix a break with a job. Tell your neighbor what percentile ranks begin and end the ‘average range.’ Discovery learning. Thanks for the handout, Dave. How soon can we break? meta.ppt

The Curve meta.ppt

Note: Other Bell-shaped Curves meta.ppt

Let’s go to the races. meta.ppt

Back to Meta-Analysis B Recall the description of meta-analysis. In meta-analysis, a researcher examines many studies on a particular strategy and derives a numerical indicator of the relative effectiveness of the strategy, averaged across all studies. This indicator is called an effect size (ES). Such an analysis of research not only provides a numerical indicator of the relative effect of a particular intervention, but the effect size (ES) also allows comparison with other approaches used in special education or related services. Suppose we had an intervention like giving vitamins (A) that was being compared to another intervention (B) like giving candy. Suppose that the groups who received vitamins (A) and the groups who received (B) “candy” performed equally at the start on a math test. meta.ppt A

Further, suppose that we monitored the progress of both groups over time, and the treatments were provided with fidelity. A B Time + 1 Month Start + 2 Month + 3 Month The two collections of groups are equal. Math Scores meta.ppt

Further, suppose that we monitored the progress of both groups over time, and the treatments were provided with fidelity. A B Time + 1 Month Start + 2 Month + 3 Month After one month, we see this pattern. Math Scores meta.ppt

Further, suppose that we monitored the progress of both groups over time, and the treatments were provided with fidelity. A Time + 1 Month Start + 2 Month + 3 Month After two months, we see this pattern. Math Scores meta.ppt B

Further, suppose that we monitored the progress of both groups over time, and the treatments were provided with fidelity. A B Time + 1 Month Start + 2 Month + 3 Month After three months, we see this pattern. Math Scores meta.ppt

If this was the result, which method would you choose to use in your classroom, A or C? A B Time + 1 Mon th Start + 2 Mon th + 3 Mon th Math Scores meta.ppt

A vertical line can be drawn to compare the relative positions of the means of the two treatments. Note that method A’s mean is actually 1 SD ahead of the B. A B Time + 1 Mont h Start + 2 Month + 3 Mont h Math Scores meta.ppt

Effect size (ES) is based on aggregated data A B or C Time + 1 Month Start + 2 Month + 3 Month Math Scores When several studies create the pattern we see here, we are more confident that the results are trustworthy (valid). meta.ppt

Effect size (ES) is based on aggregated data A B or C Time + 1 Month Start + 2 Month + 3 Month Math Scores Since the performance of group A is +1 SD beyond group B, we say that method A (giving vitamins) has an effect size of +1. meta.ppt

Is it always an even number like 1, 2, 3? The answer is ‘no.’ Effect sizes can be fractions (e.g., ES = +.5 (half an SD) or ES = +.35). It could also be negative (ES = -.6). Generally ESs that are worth paying attention to as a teacher are +.40 or higher. The highest EF that I know is 1.62 for using mnemonics as an instructional tool. Any questions from the class? meta.ppt

Summary Meta-analysis aggregates data from several studies that deal with a particular treatment. The number that describes how well the treatment works is called effect size (ES). ESs can be positive (+) or negative (-). Generally the bigger the ES for the treatment, the more you can take it to the bank. Distribute Forness’ article on meta-analysis Display some sources of information that are based on meta-analysis. meta.ppt