1. LAYING THE FOUNDATION: GARNERING SUPPORT FOR EVALUATION
Leslie A. Fierro, MPH – Independent Evaluation Consultant (SciMetrika, LLC)
&
Michele Mercier, MPH - Public Health Advisor (Asthma Program/APRHB/NCEH)
AEA Annual Conference, 2009

2. Outline
What is asthma?
Addressing Asthma from a Public Health Perspective
The burden of asthma
National Asthma Control Program
History of evaluation: State Asthma Programs
States: RFP/FOA language & capacity
Federal level: Available support
ECB Model: Motivating reasons or “triggers”

3. What is Asthma? Chronic disease of the airways that may cause:
Wheezing
Breathlessness
Chest tightness
Nighttime or early morning coughing
Episodes usually associated with widespread, but variable, airflow obstruction within the lung that is often reversible either spontaneously or with treatment

4. Pathology of Asthma Cause unknown Public health efforts focus
Normal
Asthma
Cause unknown
Public health efforts focus
on management
When asthma is properly
managed: Individuals
can lead normal, healthy,
productive lives
This drawing is from the National Heart, Lung and Blood Institute

5. National Burden of Asthma
Nationally, between 2001 & 2003 on average:
20,047,000 individuals reported having asthma*
1,772,200 emergency department visits for asthma
504,000 hospitalizations due to asthma
4, 210 deaths
*current asthma; also note estimated annual average
Centers for Disease Control & Prevention. [National Surveillance for Asthma-United States, ]. Surveillance Summaries, Oct. 19. MMWR 2007; 56(No.SS-8).

6. National Asthma Control Program
FY09: Budget Allocation
Location:
Centers for Disease Control & Prevention
National Center for Environmental Health
Division of Environmental Hazards & Health Effects
Air Pollution & Respiratory Health Branch

7. State Asthma Programs
Initially (1999): 4 state health departments funded
Currently: 34 states, D.C., & Puerto Rico funded
What they do:
Maintain & enhance statewide partnerships
Conduct asthma surveillance
Identify, prioritize & implement interventions
Focus: Asthma management
Evaluation: Focus changed over time

8. History of Evaluation: State Asthma Programs (RFP):
Evaluate all activities and document lessons learned.
Evaluate each intervention activity and the program as a whole; document lessons learned.
For all activities, develop and implement an evaluation plan which measures the effectiveness of your activities involved in each step indicated and document lessons learned.
Tone:
General, but comprehensive
No state asthma program evaluation staff

9. History of Evaluation: State Asthma Programs (RFP):
Develop and implement a plan to monitor program performance that measures the effectiveness of your program as a whole, as well as each intervention.
At a minimum, plan to assess at least one (or two) activity (activities) in each major program component in each budget year.
Tone:
Still comprehensive, but much more specific
No state asthma program evaluation staff

10. History of Evaluation: State Asthma Programs (Federal support)
Capacity at the national level:
No formalized internal team to support evaluation
Project officers provided technical assistance on as needed basis
Small & ongoing effort to develop asthma program specific evaluation guide to supplement CDC Self-Study Guide

11. Current Status of Evaluation: State Asthma Programs (RFP):
Evaluation Planning ( )
Strategic evaluation plan
Some individual evaluation plans
Evaluation Implementation & Refinement ( )
Implement individual evaluation plans
Revise, refine & update strategic evaluation plan
Develop new individual evaluation plans
Tone:
Strategize, prioritize, build evaluation capacity
½ FTE Evaluator

12. Current Status of Evaluation State Asthma Programs (Federal support)
Capacity at the national level:
Formal evaluation team
Subset of team provides direct support to states
Evaluation contract support
Develop evaluation professional development materials
Further develop evaluation skills in house

13. Or at least, how did we start down this road?
How did we get here?
Or at least, how did we start down this road?

14. “…there is usually some ‘trigger’ or motivating reason that the organization or system decides to build the evaluation capacity of its members.” (Preskill & Boyle, 2008, p. 446)

15. Motivating Reasons External: Internal:
Desire to meet accountability requirements
Change in external environment (e.g., economy)
Internal:
Changes in organization (e.g., reorg., new leader)
Perceived lack of internal evaluation knowledge & skills
Desire for new or increased funding
Desire to use evaluation to make program improvements
Selected from Preskill & Boyle, 2008, p. 446

16. Motivating Reasons These motivations…
Laid the foundation for evaluation capacity building
Set the stage for ECB at national and state levels
Still exist & likely facilitate sustainability of the effort
Once the foundation was laid, we began constructing the 1st floor…