Process Eval Quality assurance and process evaluation

Process Eval Quality –appropriateness of a set of professional procedures for a problem & objective to be achieved

Process Eval Quality control –methods used to produce documentation of the quality of program procedures

Process Eval Performance standard –minimum acceptable level of performance set by experts to judge quality of a person or program’s practice

Process Eval QA review –documentation of the competence of the program/provider with changes to improve if found inadequate

Purpose Monitoring Is it reaching the target audience? What’s effective? What’s relative costs? What needs more work?

Process Eval Methods Which methods of data collection already mentioned could you use in process eval & monitoring?

Methods of Process Eval Form used to monitor, Fig. 4.1, Windsor & Percentage exposed data next Table 4.5, Windsor

Process Eval Percentage exposed –number who come to program divided by potential number

Fig. 4.1 Form for process evaluation

Table 4.5 Percentage of participants exposed to program

Process Eval Using performance standards & calculating an ‘Implementation Index’ & ‘Program Implementation Index’

Process Eval See handout Table 4.7, Windsor - Hypothetical process evaluation of employee participation in a health promotion program

Process Eval Percentage Reached - B/A = C Performance Standard - the % that should have been reached

Process Eval Implementation Index - C/D = E If less than 1 - what does that mean?

Process Eval Program Implementation Index Sum of all E/total number of E What is the PII of this program?

Process Eval Before start a program determine the performance standards, II and PII acceptable Helpful for monitoring program

Process Eval This health program recorded a 20% behavior change rate Only 20 out of 100 enrollees changed behavior

Process Eval What phases of the program may have been a reason for the low behavior change rate? Is a PII of.78 a high enough number?

Process Eval If this program director had been monitoring the program, when should changes have been made?

Process Eval What else could have been done?

Process Eval Monitoring use of information services

Process Eval Audience reach with a message campaign

Process Eval Records of staffing & workloads (Moving to the Future, read this) Questions about how the participants likes the teacher or facilities

Methods of Process Eval Type III error –failure to implement a health education program the way intended –if outcomes evaluation shows no effect, maybe because not implemented

END Process Eval Questions?