1. Evaluation of Programs
Lisa A. Gulla MAE, MPH, HO

2. Objectives of session
To understand the difference between a goal and objective
To understand the PDSA model and its use in developing/evaluating your workplan
To understand and properly use SMART objectives
To understand and properly utilize performance management/process evaluations

3. Goals vs. Objectives Goals –
Long term – Big picture usually 5 to 10 years
To reduce the current number of breast cancer deaths by 50% in the US by 2026
Intermediate goal – Also bigger picture but generally impact is within 3 to 5 years
To reduce breast cancer mortality by addressing disparities, increasing access to quality/timely care, and improving the outcomes using patient navigation (workplan)
Objectives – How do you get there from here?
Tasks/Actions to achieve intermediate goal that can be measured.

4. Key Terms in Evaluation
SMART Objectives:
Specific
Measurable
Achievable/Attainable
Realistic
Time-Bound
Performance Management
Ongoing process of communication between staff (supervisor/employee, team approach, grantor/grantee) that occurs throughout a designated time period (usually a year but can’t be a grant period), in support of accomplishing objectives of the organization/grant
Quality Improvement
A formal approach to the analysis of performance and systematic efforts to improve it.
FADE – Focus, Analyze, Develop, Execute
PDSA – Plan, Do, Study, Act

5. Using PDSA to guide your evaluation process

6. Workplan
Key to PDSA
It allows you to map out each step of the model (objectives)
Can be used as a guide to setting up your evaluation methods/performance management
A map for whether you need to change or reevaluate your methods/data

7. Plan Becomes your Objectives in the Grant Who will do What Where
Agency, specific staff, specific team of staff
What
Education, screening, behavior change, knowledge improvement
Target Population
Where
On site, outreach in community
By When
End of year, within six months, monthly, quarterly
For how many, change by how much
Number of people educated, number or target population involved in screenings/educational programs, number of people referred for follow-up/treatment, etc., number that follow through on recommendations.

8. DO Carry out the plan – Implement the Program Document activities
Successes
Problems
Unexpected results
Begin Analysis of Data – Performance Management
What tools are you using to get the data you need
Pre/Post Tests
Data Logs (excel or other software based spreadsheet)
Sign-In sheets
Calendar of events

9. Outcome Evaluation Data required by Grantee - Workplan Demographics
As a grantee this must be collected and submitted
Process Evaluations – reflective of your stated objectives
Reporting metrics (Appendix A)
Outcome Evaluations – reflective of your stated objectives
Monitoring/Evaluation (pages 15/16 of RFA)

10. Study Complete Analysis of Data
Compare current results to predictions (initial objectives set)
Use your tools for analysis, what do they say?
Summarize what was learned - (going to laundromats on Wednesday afternoon’s yielded 30% of target population for outreach, going on Saturday’s yielded 10%)

11. Act What changes, if any, need to be made before next assessment?
To reach objectives/grant requirements

13. Successful Workplans Make sure that their objectives include:
Being SMART
Designates how it will record metrics to track/measure progress
Leaves out the evaluation tools/methods to collect and analyze data needed to measure progress/success

14. Practice Time!
Using the Project Priorities, devise objectives and an evaluation plan
Address the lack of awareness about and existence of clear, consistent, accurate, and complete breast health and breast cancer information and existing community resources in order to empower patients to more effectively navigate the continuum of care.
Meet the unique needs of minorities and other at-risk populations and decrease barriers to care by utilizing tailored messaging and strategies as well as partnerships with trusted community institutions to provide culturally and linguistically appropriate educational programs.
Develop collaborative solutions for increasing access to providers at all phases of the Continuum of Care.
Enhance provider and patient communication as it relates to linguistic challenges as well as the importance of connecting with a primary care home as a regular source of care.
Address the awareness about and availability of transportation to screening, diagnostic, and treatment services in order to improve access to vital services integral to improving health outcomes.
Provide support for services beyond screening, including diagnostics and out-of-pocket costs (including co-pays, deductibles, prescriptions, and premiums) for un- and under-insured, working poor populations in order to decrease disparities in care.
Support financial navigation for those needing financial assistance in order to decrease barriers to care in un- and under-insured populations.
Support free mammography screenings to decrease barriers to care and improve health outcomes through early detection.
Provide support for special vulnerable populations identified as experiencing extensive barriers to care, including financial difficulties.