INVITATIONAL PRACTICUM IN METAEVALUATION Session 6, 10/13/11 (Dan) 1.India’s Education Capacity Building Program & its evaluation 2.DLS metaevaluation 3.Chris Coryn’s comments re. the 2 metaevaluation reports, employment of JC Standards, uses of the metaevaluations, and next steps 4.DLS employed checklist and contents of his metaevaluation report 5.Chris Coryn’s reflections on checklist-based metaevaluation and the more general approach by Owens 6.DLS comment on applying checklists 7.Student updates on finding a metaevaluation report 8.Sample questions for next week’s test 1

India’s Education Capacity-building Program (Chris) Overview of the program Roles of Coryn and Cousins Financing of the reform and their evaluations Evaluation arrangements Origin of the metaevaluation Rationale for using the Joint Committee Standards? Selection of 3 metaevaluators Involvement of Cambridge Education Ltd. Charge to the metaevaluators 2

DLS Orientation to the Metaevaluation Assignment (Dan) Each metaevaluator should proceed independently Tailoring of a checklist to the assignment Issue re. draft status of new JC Standards Obtaining and reviewing materials Prohibition against interviewing the evaluators Production of 4 reports Meeting with some of the Indian evaluators Next steps: summative metaevaluations 3

External Consultant’s Perspective on the Process (Chris) Review of the Owens report Resignation of the India-based metaevaluator Contrast of the Owens and Stufflebeam reports Fit of the JC Standards to the Indian context Progress of the evaluation work in India Impact of the India-based evaluations Impact of the Owens and Stufflebeam reports Next steps in the evaluations and metaevaluations 4

Checklists (DLS) Manual Checklist – Filled in using Microsoft Word Automated Checklist – Microsoft Access – Excel 5

Q & A re: AADHAR METAEVALUATION REPORT (Dan) Cover sheet Acroyms page Table of Contents Executive Summary Acknowledgements Intro (particulars, documents) Description of subject evaluation plan Checklist findings (Utility, Feasibility, Propriety, Accuracy, Evaluation Accountability) Tables of results (bottom-line, for each criterion/standard, summary of +s & -s) Conclusions (nature of plan, the evaluators, Utility, Feasibility, Propriety, Accuracy, Evaluation Accountability, strengths & weaknesses, suggestions, bottom-line conclusions) Appendixes (documents reviewed, metaevaluation procedures followed, metaevaluation plan, metaevaluator’s attestation re. meeting metaevaluation criteria, metaevaluator’s credentials) 6

REFLECTIONS (Chris) Checklist-based metaevaluations General, perceptual approach applied by Owens Use of automated checklists 7

STUDENT UPDATES (Dan) Locating metaevaluation reports Locating evaluation reports 8

MID-TERM EXAMINATION (Chris) Sample test questions – Modified true-false – Multiple choice – Work sample followed by multiple choice – Matching – About 60 items Reference to distributed test plan 9