1. Jing Wan

2.  Introduction  Logic model  Evaluation methods(Design, data collection, analysis)  Communication & dissemination

3.  Evaluation questions Formative evaluation Summative evaluation  Data collection methods & Measures Formative evaluation Summative evaluation (Qualitative measures+ quantitative measure)  Communication & Dissemination

4.  Overview of logic model

6.  Inputs: Training structure & set-up Training process Training content Trainer Trainee Supportive environment

7.  Outputs & Outcomes Program products & activities Changes in trainees’ KSA Improved workforce & organizational performance

8.  Evaluation questions  Evaluation design  Data collection methods

9.  Related to inputs Is training content relevant to the trainees’ job assignment? How satisfied are the trainees with their involvement in the training session?  Related to outputs Does the training program provide the planned assignments?

10.  Related to outcomes/impacts Did the trainees experience any changes in their KSA, and job performance? What are the effects of the training program on each department of DHSMV (eg. Improved work efficiency, enhanced networking, and partnerships)?

11.  Formative (process) evaluation design How well the training program matches the evaluation theory behind its design, what the program actually does during the process.  Summative (outcome) evaluation design How the training program is related to changes in the trainees, whether the program causes the desired changes.

12.  Formative (process) evaluation Quantitative measure: What to measure: trainees training experience How to measure: after training survey From whom collected: trainees How to analyze: statistical analysis for quantitative data

15.  Formative (process) evaluation Qualitative measure: What to measure: the training meets the trainees’ expectation How to measure: after training survey From whom collected: trainees How to analyze: content analysis for qualitative measure

17.  Summative (outcome) evaluation Qualitative & quantitative measures What to measure: short-term outcomes How to measure: follow-up questionnaire & telephone interview From whom collected: trainees & subordinates How to analyze: statistical analysis & content analysis

20.  Communication challenges General evaluation anxiety (results can affect decisions) Disengagement Management operating style (resistant to change, rapid staff turnover, dysfunctional information-sharing system)

21.  Communication methods Written reporting Final evaluation report Progress reports Verbal presentations Debriefing meetings Bring together stakeholders to present key evaluation findings and recommendations or other evaluation components.