1. RET 20 03 conference An External Evaluation of the RET Program in the Materials Science and Engineering Department, University of Arizona: Our Model & Instruments Copyright 2003 © ChangeLeaders Consulting & Research Group of Arizona, Inc.

2. Grateful Acknowledgement  PI & CO-PI Supapan Seraphin & Gary Chandler for their enthusiasm and willingness to work with the evaluators as a collegial team to produce the best possible evaluation Copyright 2003 © ChangeLeaders Consulting & Research Group of Arizona, Inc.

3. Data Collection & Analysis Results & Report February 2004 In Process

4. Overview: Evaluation Model Robust model: quantitative + qualitative Multiple methods of data collection and analysis Copyright 2003 © ChangeLeaders Consulting & Research Group of Arizona, Inc.

5. Purpose: To Evaluate the effectiveness of the methods implemented to recruit and select teachers for this program; describe the characteristics of the teacher participants selected for this program; determine the nature and extent of change of the teacher participants’ knowledge, attitudes, values and beliefs about the summer research experience; and determine the extent to which the RET summer research program achieved its intended goals Copyright 2003 © ChangeLeaders Consulting & Research Group of Arizona, Inc.

6. Intended RET Program Goals impact teacher participants’ understanding of the nature of science and engineering set within Materials Science and Engineering, increase their confidence in areas of science, engineering, and technology, translate their research experience into classroom applications, and become an active player in the research group. Copyright 2003 © ChangeLeaders Consulting & Research Group of Arizona, Inc.

7. Evaluation Questions What is the nature and extent of impact of the RET program upon teachers’ science/engineering knowledge, skills, attitudes, beliefs, and proficiencies? What evidence is there that teachers transfer science knowledge, skills, attitudes, beliefs, and/or proficiencies to their science curriculum, instruction, and/or pedagogy? In the case where it is demonstrated that teachers transfer science knowledge, skills, attitudes, beliefs, and/or proficiencies to their classrooms, what outcomes do their students demonstrate?

8. Evaluation Methods: Quantitative RET Student Questionnaire Impact of the RET teachers’ research experience upon their students’: attitudes toward science (II) inquiry-based or problem-based model of learning (III) pursuit of science as a career (IV) values regarding science (V) science achievement (VI) Copyright 2003 © ChangeLeaders Consulting & Research Group of Arizona, Inc.

9. Evaluation Methods: Quantitative RET Student Questionnaire 27-items 4-Strongly Agree, 3-Agree, 2-Disagree, 1- Strongly Disagree, 0-Not Sure I like to read about science. (II) I like to work in groups to solve classroom assignments. (III) I want to be a scientist of some kind when I grow up. (IV) I think it’s important to study science. (V) I can do science. (VI) Copyright 2003 © ChangeLeaders Consulting & Research Group of Arizona, Inc.

10. Evaluation Methods: Quantitative Student Achievement Students’ performance on the RET- based lesson’s lab activities, related classroom tests or other assessments, and homework-related evidence of student performance in the RET-based curriculum. Copyright 2003 © ChangeLeaders Consulting & Research Group of Arizona, Inc.

11. Transfer of Teachers’ Knowledge and Skills to Students: Video- taping and Assessment Rubrics In RET, teachers developed a RET-based curriculum and curriculum delivery method(s) to transfer content knowledge and research skills gained in their summer experience to their students. Teachers now in process of teaching their curriculums. Evaluators review lesson plans and videotape teachers’ delivery of curriculum. Copyright 2003 © ChangeLeaders Consulting & Research Group of Arizona, Inc.

12. IN ACTION: RET Teacher & Curriculum Scanning Electron Microscope [9 th grade Earth Science Class, teacher delivering RET lesson in the scanning electron microscope] Copyright 2003 © ChangeLeaders Consulting & Research Group of Arizona, Inc.

13. Videotaping and Assessment Rubrics Independent of one another, the evaluation team of three views each videotape and assesses teacher-developed curriculum and instruction and students’ engagement in the lesson and lab activity using rubrics. When each evaluator has completed the assessment, the evaluators convene to discuss and determine a single rubric that best describes the teacher’s performance and students’ engagement. Copyright 2003 © ChangeLeaders Consulting & Research Group of Arizona, Inc.

14. Videotaping and Assessment Rubrics Rubrics developed to assist in assessing the extent to which the curriculum delivered in the lesson matches the intended curriculum as the teacher developed in the summer intended goals of the MSE RET program, and the effects upon the students. These rubrics addressed the following questions: Copyright 2003 © ChangeLeaders Consulting & Research Group of Arizona, Inc.

15. Videotaping and Assessment Rubrics To what extent did the lesson that the teacher delivered match the lesson plan that she/he developed in the RET summer research experience? To what extent did the delivered lesson demonstrate the teacher’s knowledge and skills acquired in the RET summer research experience? Copyright 2003 © ChangeLeaders Consulting & Research Group of Arizona, Inc.

16. Videotaping and Assessment Rubrics To what extent were the intended learning objectives for this lesson achieved? What are the effects and extent of effects upon student participation and engagement in the classroom activity/activities during this lesson? How clearly did the teacher communicate to students that the delivered lesson resulted from her/his participation in the RET summer research experience? Copyright 2003 © ChangeLeaders Consulting & Research Group of Arizona, Inc.

17. Assessment Rubric 5-No differences exist between the intended and actual curriculum content and between the intended and actual curriculum delivery method(s). 4-Differences exist between the intended and actual curriculum delivery method(s) but not between the intended and actual curriculum content. Copyright 2003 © ChangeLeaders Consulting & Research Group of Arizona, Inc.

18. Assessment Rubric 3- Differences exist between the intended and actual curriculum content but not between the intended and actual curriculum delivery method(s). 2-A few differences exist between the intended and actual curriculum content and the intended and actual curriculum delivery method(s). 1-No match exists between the intended and actual curriculum content and the intended and actual curriculum delivery method(s). Copyright 2003 © ChangeLeaders Consulting & Research Group of Arizona, Inc.