Adding the Second Story: Building Evaluation Capacity in the Field Carlyn Orians and Joanne Abed

Preskill H and Boyle S, 2008. A multidisciplinary model of evaluation capacity. American Journal of Evaluation, 29, 443-459.

Transfer of Learning (Who) Evaluation team Program staff CDC Evaluation Team Evaluation Contractor (Battelle) CDC Branch staff (APRHB) Programs in geographically diverse locations (state programs) Intentional Process

Transfer of Learning (What) Evaluation team Program staff (CDC, state) CDC evaluation framework Asthma program impact model Evaluation tools Strategic planning process Program structure, activities, and goals Evaluation experience and skills Pilot testing and feedback Intentional Process

A “How to” Manual Guides states through process of developing evaluation plans Promotes evaluative thinking and builds capacity Promotes strategic thinking Provides common understanding between funder, state programs, partners

Evaluation Tree Visual image to convey evaluation as….. a central part (or trunk) of our programs essential to learning and improving our programs a dynamic process of growth and change a contributor to a better environment for people with asthma

Modular Design Core Module Supplemental Modules (under development) Evaluation at APRHB in context CDC Evaluation Framework Evaluation Standards Impact model for asthma program “How to” …. strategic evaluation plan “How to” …. individual evaluation plan Supplemental Modules (under development) Surveillance Partnerships Interventions

State Asthma Program Impact Model State Asthma Program Activities Short-term outcomes Intermediate outcomes Long-term outcomes Maintain & enhance statewide asthma surveillance activities Increased awareness of asthma burden, disparities, statewide asthma efforts, & ability to manage asthma Reduced production of & exposure to triggers Asthma mortality decreased Improved asthma management behaviors of individuals… Who have asthma & their families Who provide services in settings where persons with asthma live, work, & receive medical care Build, maintain, & enhance statewide asthma partnerships Asthma disparities decreased Improved knowledge & understanding of appropriate asthma management practices & effective public health strategies related to asthma management Identify, prioritize, & implement interventions to decrease disparities & reduce state and national asthma burden Improved medical management of asthma Asthma symptoms & morbidity decreased Improved productivity & quality of life Public & organizational policies supportive of asthma management practices proposed and adopted Coordinate statewide asthma activities through creation, implementation, & revision of statewide asthma plan Improved attitudes toward asthma management practices & statewide asthma efforts Increased funding to support asthma activities Evaluate state asthma program & modify statewide plans & activities based upon findings Improved skills in asthma management and partnership functioning Statewide asthma efforts sustained & improved Improved use of available resources Increased coordination of asthma-related efforts across the state New or strengthened relationships & networks Improved infrastructure & public health practice Share findings from surveillance & evaluation efforts

Adult Learning Strategies Learning objectives Vignettes to model the planning process Explicit links to the CDC Framework Templates and models Adaptable to an on-line format with hyperlinks to glossary and detailed explanations

Learning Objectives

Vignettes

Hyperlinks

Measuring Evaluation Capacity Self-assessment instrument to measure capacity Skills, training, outlook, and experience of staff responsible for evaluation AND Experience and support from organizations Uses: Formative: direct training and assistance Outcome: measure results of capacity- building efforts