1. Effects of Feedback in a Computer-Based Learning Environment on Students’ Learning Outcomes : A Meta-Analysis Fabienne M. Van der Kleij, Remco C. W. Feskens, Theo J. H. M. Eggen. Review of Educational Research Month 201X, Vol. XX, No. X, pp. 1–37 DOI: 10.3102/0034654314564881 © 2015 AERA. http://rer.aera.net

2. Outline Abstract Type of Feedback Evidence of the Effects of Feedback in Computer- Based Environments Gaps in the Literature Hypotheses Method Results Discussion

3. Abstract 1 1. From 40 studies, 70 effect sizes were computed, which ranged from -0.78 to 2.29. 2. Elaborated feedback (EF; e.g., providing an explanation) produced larger effect sizes (0.49) than feedback regarding the correctness of the answer (KR; 0.05) or providing the correct answer (KCR; 0.32). 3. EF was particularly more effective than KR and KCR for higher order learning outcomes. 4. Effect sizes were positively affected by EF feedback, and larger effect sizes were found for mathematics compared with social sciences, science,and languages.

4. Abstract 2 5. Effect sizes were negatively affected by delayed feedback timing and by primary and high school. 6. The results suggested that immediate feedback was more effective for lower order learning than delayed feedback.

5. Type of Feedback ： KR KR(knowledge of results) indicates whether the answer is correct or incorrect but does not provide the correct answer or any additional information. Its main purpose is to reinforce the correct recall of facts, which is consistent with a behaviorist view of learning. Over time research has recognized that a trial-and-error procedure was not very effective in student learning because it does not inform the learner about how to improve.

6. Type of Feedback ： KCR KCR(knowledge of correct response) its main purpose is to revise the student’s incorrect responses by providing the correct answer. both KR and KCR merely have a corrective function.

7. Type of Feedback ： EF EF(elaborated feedback) the process itself takes on the forms of new instruction, rather than informing the student solely about correctness. can take many forms, such as hints, additional information, extra study material, and an explanation of the correct answer. often accompanied by KCR or KR.

8. Evidence of the Effects of Feedback in Computer-Based Environments1 1. Computer-based assessments (CBAs) more timely feedback, automated scoring, and higher test efficiency. serve to immediately resolve the gap between students’current status in the learning process and the intended learning outcomes. assist teachers in providing students with individualized feedback. 2. Azevedo and Bernard (1995) mostly aimed at lower order learning outcomes. (22 studies included) 0.80 for an immediate posttest, 0.35 for a delayed posttest. did not take any feedback characteristics (such as type) into consideration. based on the small sample, the datedness of the studies included and the lack of identifying moderator variables.

9. Evidence of the Effects of Feedback in Computer-Based Environments2 3. Jaehnig and Miller (2007) conducted a systematic review of the effects of different feedback types in programmed instruction. (from 1964 to 2004, N = 33) KR is not effective for learning,that KCR is effective sometimes, and that EF seems to be the most effective. the large number of outdated studies and the different approach. 4. Van der Kleij et al.(2011) the results found in various experiments on the effects of written feedback in a CBA were compared. KR seems to be ineffective, KCR moderately beneficial for obtaining lower order learning outcomes and EF beneficial for obtaining both lower order and higher order learning outcomes.

10. the results suggest that feedback can positively affect learning and indicate that EF is generally more effective. the results of Jaehnig and Miller (2007) apply only to lower order learning outcomes. As well, the systematic review by Van der Kleij et al. (2011)focused only on the effects of written feedback and used strict inclusion requirements. they do not shed sufficient light on the complex interplay of feedback type, feedback timing, and the level of learning outcomes and do not give usable insights into the magnitude of the effects. there is a need for an updated meta-analysis that takes into account variables that seem relevant based on the literature and that builds on some conclusions drawn in existing review studies. Gaps in the Literature

11. Hypotheses KR and KCR have a small to moderate positive effect on lower order learning outcomes. KR and KCR have virtually no effect on higher order learning outcomes. EF has a moderate to large positive effect on lower order learning outcomes. EF has a moderate to large positive effect on higher order learning outcomes. there is an interaction effect between feedback timing and the level of learning outcomes. According to Hattie(2009), an effect size of 0.2 can be considered small, an effect size of 0.4 can be considered moderate, and effect sizes of 0.6 are classified as large.

12. The primary search included the online databases ERIC, PsycInfo, ScienceDirect, Scopus, and Web of Science. studies to be included had to meet the following criteria: (a) the study was published in a journal article, scientific book, book section or dissertation. (b) the study was published in English language. (c) the study compared the effects of different types of feedback in a computer-based assessment. Method ： Data Collection1

13. Example of eligible experimental and control conditions ： Method ： Data Collection2 1. The experimental group and control group took an identical formative test. 2. The experimental group received immediate EF on each item. 3. The control group received immediate KR on each item in the formative test. 4. Both groups took the same posttest on the same content as the formative test.

14. The study investigated the effects of immediate KCR, immediate EF, and no feedback on items integrated into lesson material. Three effect sizes computed ： (a) the effects of KCR versus no feedback. (0.93) (b) EF versus no feedback. (0.99) (c) EF versus KCR. (0.92) The full-text versions of studies that met the inclusion criteria were retrieved. Method ： Data Collection3

15. Coding was conducted independently by four researchers. The agreement rate between the coders was.81 for all studies and.81 for all effect sizes. In some studies with multiple experimental conditions, participants received the same type of feedback under different conditions. Method ： Coding Procedures

16. Using information from several studies creates the opportunity to provide a structured summary of a specific research topic and to find relationships between variables that otherwise would not be detected. Effect sizes in this meta-analysis express the effectiveness of each study’s variable of interest between a treatment and control group in terms of SD units. Usually, estimates of effect sizes obtain a plus sign if the treatment group is better than the control group. Method ： Statistical Methods1

17. Method ： Statistical Methods2 (1) (2) (3) (4) We used the standardized mean differences effect size statistic. (1) (2) (3) (4) This statistic uses the contrast between the treatment and control groups divided by the pooled SD. An unbiased alternative estimator was used to correct for this bias. This estimator has the following standard error:

18. Method ： Statistical Methods3 (5) (6) (5) (6) Optimal weights are constructed using the standard error of the effect size (used in the fixed effects models) In random effects models these weights are supplemented by a constant that represents the variability across population effects. In order to assess whether studies with different characteristics also have different mean effect sizes, we conducted an analysis of variance for effect sizes. To establish simultaneously the relationships between effect sizes and several study characteristics, we performed a weighted regression analysis.

19. 使用 gcc 跨平台編譯的範例圖 array_sum.c gcc array_sum_IA32.s arm-elf-gcc array_sum_ARM.s arm-elf-gcc gcc array_sum_ARM.o array_sum_IA32.o