LAYING THE FOUNDATION: GARNERING SUPPORT FOR EVALUATION Leslie A. Fierro, MPH – Independent Evaluation Consultant (SciMetrika, LLC) & Michele Mercier, MPH - Public Health Advisor (Asthma Program/APRHB/NCEH) AEA Annual Conference, 2009

Outline What is asthma? Addressing Asthma from a Public Health Perspective The burden of asthma National Asthma Control Program History of evaluation: State Asthma Programs States: RFP/FOA language & capacity Federal level: Available support ECB Model: Motivating reasons or “triggers”

What is Asthma? Chronic disease of the airways that may cause: Wheezing Breathlessness Chest tightness Nighttime or early morning coughing Episodes usually associated with widespread, but variable, airflow obstruction within the lung that is often reversible either spontaneously or with treatment

Pathology of Asthma Cause unknown Public health efforts focus Normal Asthma Cause unknown Public health efforts focus on management When asthma is properly managed: Individuals can lead normal, healthy, productive lives This drawing is from the National Heart, Lung and Blood Institute

National Burden of Asthma Nationally, between 2001 & 2003 on average: 20,047,000 individuals reported having asthma* 1,772,200 emergency department visits for asthma 504,000 hospitalizations due to asthma 4, 210 deaths *current asthma; also note estimated annual average Centers for Disease Control & Prevention. [National Surveillance for Asthma-United States, 1980-2004]. Surveillance Summaries, Oct. 19. MMWR 2007; 56(No.SS-8).

National Asthma Control Program FY09: Budget Allocation Location: Centers for Disease Control & Prevention National Center for Environmental Health Division of Environmental Hazards & Health Effects Air Pollution & Respiratory Health Branch

State Asthma Programs Initially (1999): 4 state health departments funded Currently: 34 states, D.C., & Puerto Rico funded What they do: Maintain & enhance statewide partnerships Conduct asthma surveillance Identify, prioritize & implement interventions Focus: Asthma management Evaluation: Focus changed over time

History of Evaluation: State Asthma Programs (RFP) 1999-2005: Evaluate all activities and document lessons learned. Evaluate each intervention activity and the program as a whole; document lessons learned. For all activities, develop and implement an evaluation plan which measures the effectiveness of your activities involved in each step indicated and document lessons learned. Tone: General, but comprehensive No state asthma program evaluation staff

History of Evaluation: State Asthma Programs (RFP) 2006-2009: Develop and implement a plan to monitor program performance that measures the effectiveness of your program as a whole, as well as each intervention. At a minimum, plan to assess at least one (or two) activity (activities) in each major program component in each budget year. Tone: Still comprehensive, but much more specific No state asthma program evaluation staff

History of Evaluation: State Asthma Programs (Federal support) Capacity at the national level: No formalized internal team to support evaluation Project officers provided technical assistance on as needed basis Small & ongoing effort to develop asthma program specific evaluation guide to supplement CDC Self-Study Guide

Current Status of Evaluation: State Asthma Programs (RFP) 2009-2014: Evaluation Planning (2009-2010) Strategic evaluation plan Some individual evaluation plans Evaluation Implementation & Refinement (2010-2014) Implement individual evaluation plans Revise, refine & update strategic evaluation plan Develop new individual evaluation plans Tone: Strategize, prioritize, build evaluation capacity ½ FTE Evaluator

Current Status of Evaluation State Asthma Programs (Federal support) Capacity at the national level: Formal evaluation team Subset of team provides direct support to states Evaluation contract support Develop evaluation professional development materials Further develop evaluation skills in house

Or at least, how did we start down this road? How did we get here? Or at least, how did we start down this road?

“…there is usually some ‘trigger’ or motivating reason that the organization or system decides to build the evaluation capacity of its members.” (Preskill & Boyle, 2008, p. 446)

Motivating Reasons External: Internal: Desire to meet accountability requirements Change in external environment (e.g., economy) Internal: Changes in organization (e.g., reorg., new leader) Perceived lack of internal evaluation knowledge & skills Desire for new or increased funding Desire to use evaluation to make program improvements Selected from Preskill & Boyle, 2008, p. 446

Motivating Reasons These motivations… Laid the foundation for evaluation capacity building Set the stage for ECB at national and state levels Still exist & likely facilitate sustainability of the effort Once the foundation was laid, we began constructing the 1st floor…