1. Developing Tools to Measure and Improve Public Health Preparedness
David J. Dausey, PhD

2. Ultimate Goals Move the field of public health forward
Develop metrics to reliably and validly measure preparedness over time
Create general tools that could be used as templates by public health agencies for training and assessment
Change the culture of public health
Focus more on measurement
Use of CQI principles
Move to an all hazards approach to solving problems
Encourage the involvement of more partners in public health preparedness

3. Two Examples 24/7 Proficiency Tests
Develop and test a method to measure compliance with an accepted standard
Use 24/7 responsiveness as a test case
Epi Response Tabletop Exercises
Design and test a suite of tabletop exercises as training and assessment tools for public health preparedness
Focus on detection and initial response to outbreaks as a test case
24-7--
Receipt-Did the right person receive the call in a timely manner
Response-Was the initial response timely and appropriate
TT—
Pick up where the 24/7 tests leave off--Local health department’s are the primary actors considered

4. 24/7 Proficiency Tests

5. An Example of a Standard
A LPHA should be able to receive and respond to emergency case reports 24/7/365
Have a single well-publicized telephone number
Have a phone triage protocol to process urgent case reports.
Be able to respond to urgent case reports with a trained public health professional within 30 minutes of receiving the report.
Be able to handle calls with a ‘warm transfer’

6. Challenges in Measuring Compliance with the Standard
Self-report
Most health departments report they achieve this standard
Few have actually tested it
Methods to validate self-report or measure compliance with the standard undeveloped
Unit of observation
Does the whole simply equal the sum of the parts?

7. Methods Single well publicized number?
Searched phone books, called information, reviewed web sites
Can receive/respond to calls?
Purchased disposable cell phones
Programmed to local area code
Placed 6-10 unannounced calls over 4 mos.
Asked to speak with ‘action officer’
Response by trained public health officer?
Posed as health professionals reporting a case (subset)

8. Results 143 calls to 19 LPHAs over 10 months
3 of 19 (15%) tests terminated prematurely
9 of 19 (47%) had multiple phone numbers
9 of 19 (47%) responded to all calls
2 of 19 (9%) responded to all calls with warm transfers and within 30 minutes

9. Distribution of Response Times by Testing Period

10. Tabletop Exercises

11. Test Elements Built upon response after 24/7 tests
What components do you test?
Key aspects of an outbreak investigation
Logic model
What diseases do you test around?
Danzig (2003)
Collateral benefits
Reingold (1998), Plant (1998), Gregg (1998) and others outlined the Outbreak investigation:
Establish case definition(s), Confirm cases are “real”, Establishing a background rate of disease and finding cases, Find cases, decide if there is an outbreak, define scope of the outbreak, Examine descriptive epidemiologic features of cases, Develop and test hypotheses, Collect and test environmental samples, Implement control measures, Interact with the press, inform the public,
Danzig:
Large-scale outdoor aerosol anthrax attack, Large-scale outdoor aerosol smallpox attack, Botulism toxin in cold drinks, Foot and mouth disease among livestock

12. Exercise Design
Designed to be used by health departments to evaluate their ability to mount an initial epidemiological response to an outbreak
Participants - key actors involved in public health emergencies
2, 4, or 6 hour applications of varying difficulties

13. Strengths
Health departments have come a long way since 9/11 in BT prepardeness
Increases in “regional staff”
Recognition of problem areas
Improvements to response capabilities

14. Ongoing Challenges Staff knowledge/training Communications
Interagency coordination
Surge capacity
Staffing
Who’s in charge
Connect with health care system
Handoffs
Staff knowledge: HD epidemiologist “what is active surveillance”
Communication with media example: “You scared me to death. If I heard you say that on the news, I would get my kids and leave the state”
Misleading the media example: “I’m hear to investigate a fly in the pizza”
Inability to articulate the problem example: HD director unable to briefly summarize the problem for the state epidemiologist
Assumptions and expectations example: “The state health department does that sort of thing not us”
Whose in charge example: “Law enforcement would be taking orders from public health”
Communication with hospitals examples: “I’m sure the hospitals would be communicating with one another”
Hospitals on divert example: “The hospitals have a schedule for diverting patients, we’re not involved with that”
Limited or no plans for dealing with or handling a surge of calls to the health department
Dedication example: “Our staff are very dedicated, they would come to work even if there was a risk to them personally”
Family example: “Our staff wouldn’t show up to work unless we took care of their families first [with vaccines or antibiotics, etc]”
Law enforcement example: “Law enforcement needs to be educated by public health staff because fear will keep them from coming to work”

15. Ongoing Exercise Considerations
Exercise design
Build your own exercise
Matching exercise difficulty level with health department proficiency level
Exercise presentation
How much of a role should the facilitator have (e.g., how much probing)?
How much information do we give participants during the exercise?

16. Ongoing Measurement Considerations
Performance measures
Limited evidence base
What sets of measures do you use
Scoring exercises
How to adequately quantify performance
Inter-rater reliability
After action reports
Designed around performance measures
What format is most useful for participants