What are we learning about training in evidence-based public health among local practitioners?

THANKS!! Ross BrownsonJanet Canavese Beth BakerKatie Stamatakis Katie DugganCarol Brownson Paul ErwinCase Western PRC Robert FieldsUNC PRC Carson SmithMI PRC Rodrigo ReisU of WA PRC Carolyn Leep (NACCHO)Jenine Harris Rachel TabakChris Casey Amy EylerBeth Dodson Kathleen Wojciehowski

What is Project LEAD? 4 Aims: 1.Describe the evidence-base for local EBPH in the United States (esp. A- EBPs) 2.Test the effectiveness of local-level EBPH capacity building in 4 states (Train the Trainer) 3.Describe a range of local models in EBPH 4.Translate and disseminate findings to stakeholders

Selected findings (Aim 1)

Administrative evidence- based practices (A-EBP S )  Agency (health department)-level structures and activities that are positively associated with performance measures (e.g., achieving core public health functions, carrying out evidence- based interventions) Brownson, Allen, Duggan, Stamatakis, & Erwin. Am J Prev Med, 2012

Results, 5 EBP domains l Workforce development l QI (82%), EBDM (59%) l Leadership l Staff participation (84%) hire w/ PH degree (36%) l Organizational climate and culture l Life long learning (71%), culture supports EBDM (42%) l Relationships and partnerships l Importance (92%), share resources (68%) l Financial processes l Multi funding sources (96%), QI resources (55%)

Predictors of a-EBP s Predictors of A-EBPs in the United States, 2012 (n=317) LHD Characteristic OR (95% CI)aOR (95% CI) Population of Jurisdiction <25, ,000 to 49, (3.3, 15.0)*7.5 (3.3, 17.3)* 50,000 to 99, (2.7, 12.2)*4.9 (2.1, 11.2)* 100,000 to 499, (4.3, 19.6)*7.1 (3.0, 16.9)* 500,000 or larger7.1 (3.0, 16.8)*4.4 (1.6, 12.5)*

Early Results Patterns of A-EBPs and use of EBDM vary significantly based on: 1.Population size served by the LHD (strongest predictors after adjustment) a.Smaller HDs 3X more likely to be led by a nurse 2.Governance structure (state governed) 3.Age group in the 50s

Use of resources  26% of LHDs reported systematic reviews as 1 of top 3 most important resources  34% were not familiar with the Community Guide

A tale of two worlds… How LHDs learn about research findings? How researchers perceive they most effectively reach practitioners? 1. Professional associations1. Journal articles 2. Seminars/workshops2. Face-to-face meetings 3. alerts3. Media interviews 4. Journal articles4. Press releases

Selected findings (Aim 2)

Train the Trainer November 2012, St. Louis Missouri Recruitment of PRC’s –Geographic distribution –PRCs with established relationships with PHTCs or PBRNs –“Ready to hit the ground running”

Training Locations Case Western Reserve University Prevention Research Center for Healthy Neighborhoods and PBRN University of Michigan Prevention Research Center of Michigan and Michigan Public Health Training Center University of North Carolina at Chapel Hill Center for Health Promotion and Disease Prevention and PHTC University of Washington Health Promotion Research Center and PHTC

Train the Trainer Process Partners from PRC came to St Louis for 3 day training and overview of Evidence Based Public Health course Conference calls with each site to set up logistics and provide technical assistance EBPH slides and materials for trainings given to each site PRC in conjunction with PBRN or PHTC put on training for local health departments in their area

Train the Trainer Review of Module Content Focus on “Process” for course delivery

Recruiting participants for the State Trainings s to various public health listservs –PBRN, LHD commissioners, directors of nursing, etc. Website postings –PRC and PHTC Announcements and flyers at conferences Help from state health department **Each state had waiting lists

Participants of Trainings 57% program manager or coordinator, 22% top executive, administrator, or assistant director 2% doctoral degree, 24% public health masters, 37% other masters, 9% nursing degree, 28% bachelors degree or less Mean years in current position = 5.2 years, mean years in public health = 8.8 years Number of organizations represented at state trainings in person = 101 (OH = 24, MI = 21, NC = 26, WA = 20)

Course Format Two locations used traditional in-person training and two used mixed modes for course delivery i.e. in-person and webinars for distance learning capabilities –WA – 3 one-hour webinars with 2 days of in- person training –MI – 2 onsite training days, 3 modules each day, 3 modules by webinar, 2 of which had multiple days

11 EBPH Skills Evaluated (pre-training course survey) I Can… develop a concise statement of the issue conduct a community assessment describe the issue in a quantitative way discover what is known about the issue through the scientific literature prioritize program or policy options that address the issue understand economic evaluation to use findings from others to assist with determining which intervention would be best suited for my community and population of interest understand economic evaluation to assist in the design and implementation of an economic evaluation develop an action plan for the program or policy qualitatively evaluate the program or policy quantitatively evaluate the program or policy economically evaluate the program or policy

Pre/Post Skill Abilities (course survey) Data show participants felt more capable of all 11 skills after the training –Differences in ability scores between pre and post evaluations were all statistically significant using p=.01 for sites evaluated

An Example: Skills Ability for Case Western and NC combined I can prioritize program or policy options that address the issue I can qualitatively evaluate the program or policy Number of responses Pre Post Pre Post

Views Related to EBDM (pre-training survey) I feel that I have the skills necessary for developing evidence-based interventions (No. 76, 59.4%) I feel that I can effectively communicate information on evidence-based strategies to policy matters (No. 54, 42.2%) I feel evidence-based interventions are packaged in a way that I can use them (No. 34, 26.6%) I feel evidence-based interventions are designed in a way to be self-sustaining (No. 30, 23.6%) I feel that I need to be an expert on many issues in order to effectively make evidence-based decisions (No. 22, 17.2%) My fears about job security prevent me from using EBDM (No. 6, 4.7%)

Which of the following would most encourage you to utilize EBDM in your work? (pre-training survey) Placing a high priority on EBDM in your work Trainings on EBDM Positive feedback for EBDM use

Which of the following would most encourage you to utilize EBDM in your work? Director’s Survey Trainings on EBDM Placing a high priority on EBDM by leaders in my agency Positive feedback for EBDM use Programmatic Survey Placing a high priority on EBDM in your work Trainings on EBDM Positive feedback for EBDM use

Gap between importance and availability of skills (pre-training survey) Economic Evaluation 4.1 Communicating research to policy makers 3.5 Evaluation designs 3.3 Adapting interventions 3.2 Prioritizing health issues 2.7

What participants most liked about the course (post-training course survey) The breadth of the topics truly encouraged use of evidence across all functions of a LHD, not just programs Being able to concentrate on learning and having a systematic approach to follow Presents a whole new way of thinking about public health practice

Lessons Learned Webinars – advantages and disadvantages Advantages Quicker and easier for participants Less expensive Flexibility of combined format was appreciated by participants Disadvantages Opportunity for both formal and informal interaction among participants and between participants and instructors is much less Onsite instruction was more effective

Take away messages from Aim 2 All surveys point to the need for more training in EBPH The pre/post evaluations received from the trainings indicate that participants feel more competent in EBPH skills after training EBPH trainings = improved skills, ability and increased potential for change in local health departments

Ideas going forward… l Continue analyses, featuring more complex data linkage l Explore context and meaning of Aim 1 gaps l Develop new approaches for these populations (e.g., findings for nurses) l Explore new approaches for information seeking l Enhance Aim 2 additional technical assistance (beyond the EBPH courses) l Scaling up findings from Aim 2 l Use Aim 3 findings to find and test new leverage points l Link more closely with AcademyHealth for dissemination