1. Caitlin Carpenter Sheri Denning Alan Elmore Patricia Oliver Samarah Shakir

2.  Evaluation Management  Data collection  Analysis  Results  Reporting  Stakeholders  Closing Statements  References

3. Stakeholder Name Stakeholder Category Interest or Perspective Role in the EvaluationHow and When to Engage Scott WatkinsSecondary Front-line assistance Scott is allowing NCSU Evaluators to incorporate the pre and post test materials into the seminar curriculum. Scott has already agreed to assist NCSU Evaluators in the collection of data. Samarah ShakirPrimaryProject Leader Samarah is very active in working with Scott to coordinate the inclusion of pre and post test materials into the seminar curriculum. Engaged throughout the process Caitlin CarpenterPrimary Project Team Member Caitlin is responsible for analyzing and interpreting the data collected from the evaluation Engaged throughout the process Sheri DenningPrimary Project Team Member Sheri is responsible for the management and implementation of the evaluation Engaged throughout the process Alan ElmorePrimary Project Team Member Alan is responsible for stating the evaluation purpose and identifying the project stakeholders Engaged throughout the process Patricia OliverPrimary Project Team Member Patricia is responsible for identifying how the evaluation will be designed and the data collected. Engaged throughout the process Dr. Kate GuerdatPrimaryProject CustomerDr. Guerdat will provide final approval or denial of evaluation proposal as well as feedback on results and presentation. Dr. Guerdat has already provided approval on evaluation proposal. Final draft of results due on April 26, 2013

4. IndividualTitle or RoleResponsibilities Scott WatkinsDELTA Associate Director & Seminar Facilitator Distribute and collect the pre/posttest, send out level 3 survey to seminar participants Samarah ShakirNSCU Evaluators Project Leader Manage the evaluation process and coordinate the communication between DELTA and NCSU Evaluators for the administration of evaluation instruments. Will collection of Level one and two results Alan ElmoreNCSU Evaluators Project Team Member Level three evaluation instrument Sheri DenningNCSU Evaluators Project Team Member Level two evaluation instruments Patricia OliverNCSU Evaluators Project Team Member Level two evaluation instruments Caitlin CarpenterNCSU Evaluators Project Team Member Level three evaluation instrument

5. Evaluation Questions Data Collection Method Activities Needed Person(s) Responsible Due Date Do Participants understand that the differences between Moodle 1 and Moodle 2? Pre/posttest Paper pre-testScott WatkinsMarch 21, 2013 Paper posttestScott WatkinsMarch 21, 2013 Post seminar email & Survey links Participant email addresses Scott Watkins April 4, 2013 Survey linksScott Watkins and NCSU Evaluators Team April 4, 2013 Was excitement and momentum built for the Moodle 2 upgrade Post seminar email & Survey links Participant email addresses Scott WatkinsApril 4, 2013 Survey linksScott Watkins and NCSU Evaluators Team April 4, 2013

6. Analysis to Be Performed Data to Be Analyzed Person(s) Responsible Due Date Level one SurveySurvey resultsNCSU Evaluators Team April 11, 2013 Level two Survey instrument Pre/posttest resultsNCSU Evaluators Team April 11, 2013 Level three SurveySurvey resultsNCSU Evaluators Team April 11, 2013

7.  Emailed survey after completion of seminar  6 responses received  Contained 7 Likert Scale Questions on a Strongly Agree to Strongly Disagree scale  6 questions received 100% Agree  1 question received 83% Agree and 17% Neutral  Key Evaluation Questions

8.  Pre/Post Test Design  Sample group selected by date  Results indicate learning transfer  Reported increase in level of comfort with the new software  Improvement in excitement for the new software  More than 80% of respondents can list three changes between the prior and current versions of the software.

9.  Post-session Survey through Survey Monkey  Results indicate behavioral change  Respondents continue to feel excited about the change  Respondents report an increase in knowledge and comfort level after attending the session  More than 80% of respondents discussed the training and the elements of the new software with coworkers.

10. Stakeholder NameStakeholder Category Interest or Perspective Role in the EvaluationHow and When to Engage Scott WatkinsSecondaryFront-line assistance Scott is allowing NCSU Evaluators to incorporate the pre and post test materials into the seminar curriculum. Scott has already agreed to assist NCSU Evaluators in the collection of data. Dr. Kate Guerdat PrimaryProject Customer Dr. Guerdat will provide final approval or denial of evaluation proposal as well as feedback on results and presentation. Dr. Guerdat has already provided approval on evaluation proposal. Final draft of results due on April 26, 2013

11.  Results  Questions/Comments  Please contact Samarah Shakir at sagreen@ncsu.edu

12.  (2013, March 15). Evaluation plan outline. website: https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&sqi =2&ved=0CC8QFjAA&url=http%3A%2F%2Fwww.cdc.gov%2Fasthma%2Fprogra m_eval%2FAppendixF_Evaluation_Plan_Outline.doc&ei=TXJoUfHQOYWK8QT _tIHACg&usg=AFQjCNFFOBS4wHk65mDmY7uaElpmpRSH3Q&bvm=bv.4517533 8,d.eWU https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&sqi =2&ved=0CC8QFjAA&url=http%3A%2F%2Fwww.cdc.gov%2Fasthma%2Fprogra m_eval%2FAppendixF_Evaluation_Plan_Outline.doc&ei=TXJoUfHQOYWK8QT _tIHACg&usg=AFQjCNFFOBS4wHk65mDmY7uaElpmpRSH3Q&bvm=bv.4517533 8,d.eWU  Bates, R. (2004, June 8). A critical analysis of evaluation practice: the Kirkpatrick model. Retrieved April 12, 2013, from Evaluation and Program Planning: http://aetcnec.ucsf.edu/evaluation/bates_kirkp_critique.pdf  Russ-Eft, D. “Evaluability Assessment of the Adult Education Program (AEP).” Evaluation and Program Planning 9 (1986): 39-47  Russ-Eft, D. and Preskill, H. (2009). Evaluation in Organizations, 2nd Edition. New York: Basic Books.