1. Project Evaluation Don Millard John Yu dmillard@nsf.gov zyu@nsf.govdmillard@nsf.govzyu@nsf.gov March 27, 2012 Guy-Alain Amoussou Lou Everett gamousso@nsf.gov leverett@nsf.gov gamousso@nsf.gov leverett@nsf.gov March 28, 2012 Handout 1

2.  Most of the information presented in this workshop represents the presenters’ opinions and not an official NSF position  Local facilitators will provide the link to the workshop slides at the completion of the webinar.  Participants may ask questions by “raising their virtual hand” during a question session. We will call on selected sites and enable their microphone so that the question can be asked.  Responses will be collected from a few sites at the end of each Exercise. At the start of the Exercise, we will identify these sites in the Chat Box and then call on them one at a time to provide their responses. 2

3.  Learning must build on prior knowledge ◦ Some knowledge correct ◦ Some knowledge incorrect – Misconceptions  Learning is ◦ Connecting new knowledge to prior knowledge ◦ Correcting misconceptions  Learning requires engagement ◦ Actively recalling prior knowledge ◦ Sharing new knowledge ◦ Forming a new understanding 3

4.  Effective learning activities ◦ Recall prior knowledge -- actively, explicitly ◦ Connect new concepts to existing ones ◦ Challenge and alter misconceptions  Active & collaborative processes ◦ Think individually ◦ Share with partner ◦ Report to local and virtual groups ◦ Learn from program directors’ responses 4

5.  Coordinate the local activities  Watch the time ◦ Allow for think, share, and report phases ◦ Reconvene on time -- 1 min warning ◦ With one minute warning, check Chat Box to see if you will be asked for a response  Ensure the individual think phase is devoted to thinking and not talking  Coordinate the asking of questions by local participants and reporting local responses to exercises 5

6. The session will enable you to collaborate more effectively with evaluation experts in preparing credible and comprehensive project evaluation plans …. it will not make you an evaluation expert. 6

7. After the session, participants should be able to:  Discuss the importance of goals, outcomes, and questions in the evaluation process ◦ Cognitive and affective outcomes  Describe several types of evaluation tools ◦ Advantages, limitations, and appropriateness  Discuss data interpretation issues ◦ Variability, alternative explanations  Develop an evaluation plan in collaboration with an evaluator ◦ Outline a first draft of an evaluation plan 7

8.  The terms evaluation and assessment have many meanings ◦ One definition  Assessment is gathering evidence  Evaluation is interpreting data and making value judgments  Examples of evaluation and assessment ◦ Individual’s performance (grading) ◦ Program’s effectiveness (ABET and regional accreditation) ◦ Project’s progress and success (monitoring and validating)  Session addresses project evaluation ◦ May involve evaluating individual and group performance – but in the context of the project  Project evaluation ◦ Formative – monitoring progress to improve approach ◦ Summative – characterizing and documenting final accomplishments 8

9.  Think about your favorite course. What types of in-class activities could be called: ◦ Assessment versus Evaluation ◦ Formative versus Summative Evaluation  Exercise ---- 6 min ◦ Think individually -------- ~2 min ◦ Share with a partner ----- ~2 min ◦ Report in local group ---- ~2 min  Watch time and reconvene after 6 min  Use THINK time to think – no discussion, Selected local facilitators report to virtual group  With one minute warning, check Chat Box to see if you will be asked for a response 9