A Structural HIV Prevention Intervention to Help CBOs Implement Evidence-Based Interventions Effectively Susan M. Kegeles, PhD, Gregory Rebchook, PhD, & the TRIP team CAPS/University of California, San Francisco Funded by NIMH

Translating Research Into Practice Substantial resources have been spent on developing and testing HIV prevention interventions: has resulted in a number of evidence-based interventions But until interventions are effectively implemented, the research has little impact on the epidemic This project focuses on how to get an evidence-based intervention put into practice in the “real world”

The Mpowerment Project (MP) Demonstrated effectiveness –Listed in CDC Compendium of HIV prevention interventions with evidence of effectiveness –Scientifically tested in several communities Community-level intervention with multiple components Many CBOs want to implement the MP & call for assistance, training

Time for a New Type of Intervention Previous approach to influencing epidemic: develop an intervention to help change young MSM’s sexual risk behavior (MP) Structural intervention is needed: to impact the way that HIV prevention is conducted by CBOs Now: develop an intervention aimed at helping CBOs to implement MP effectively & with fidelity to MP’s core elements and guiding principles

Developing Intervention Began developing ideas about components by observing CBOs when we were testing the MP Observed CBOs that took over MP as we would leave the field in researching it Provided help to CBOs implementing MP Developed preliminary replication package (CDC’s REP project - “Replicating Effective Programs”) & tested it with CBO

Lessons Learned for Intervention - partial list - Staff turnover is rapid Often there is no organizational memory of prevention program after staff leave –Some staff are technologically savvy; others less so –CBOs can’t always afford up-to-date technology –Need to build rapport, over time, so that trust develops –CBOs don’t want to be told exactly what to do and how to do it Need to give suggestions, framework Need to be able to modify the intervention for community and cultural context

Lessons Learned (cont.) CBOs want to hear what worked and didn’t work when we conducted research on MP Staff often don’t reach out for technical assistance (TA) early, but rather, often wait until crisis hits –Staff may not recognize when need help –Need to reach out to them, so that when crises do hit, can provide help –Or better yet - avoid crises Diverse ways of learning –visual information, didactic presentations, experiential learning - multiple methods aid in learning

This study: to evaluate how well the Mpowerment Project Technology Exchange System (MPTES) works in helping organizations implement the MP This presentation: to evaluate how the MPTES is utilized by CBOs and their attitudes towards its different components - preliminary results

Methods –Provide MPTES to CBOs and examine uptake of the various components –N = 52 CBOs’ data presented here –Will follow 70 CBOs over follow-ups, at 6 month intervals –CBOs are interviewed by phone at each assessment –This presentation: examine utilization patterns after CBOs have MPTES for at least 6 months (follow-up 1) –Study evaluator interviews EDs, project supervisors, & coordinators

The Mpowerment Project Technology Exchange System (MPTES) 7 components: –Program Manual –Program Overview Video –Small group facilitator’s manual –Small group training video –3-day training –Technical Assistance (TA) –Internet Resources (website focused on here)

Program Manual Developed after input by former staff, CBOs using it, advisory board comprised of CBOs from around SF/Bay Area Lots of visuals Carefully organized Very user friendly

Program Overview Video & Small Group Video To give a “feel” for what intervention is like Filmed on location when conducting intervention in Albuquerque Overview video: to use with funders, boards of directors, agency when deciding if to implement MP, sometimes core groups (initially) Small group video: to train small group facilitators, to be used with facilitator’s manual for small groups

Training Designed for project staff and supervisors Experiential and interactive Conducted by original researchers and former Project Coordinators 3-days long In-depth Comprehensive Information exchange among CBOs at training

Technical Assistance (TA) Collaborative Approach TA to coordinators & supervisors Pro-active –TA providers call each CBO every 2 to 3 weeks –CBOs can request TA at other times Tailored –TA helps CBOs adapt MP to the needs of their community Credible Providers –Former MP Coordinators –Experienced implementing MP both during the research period and in a CBO

Design complements the training manual Online modules –HTML –PDF Message board forum Chat room Examples of outreach materials (over 100) Links and team photos

Evaluation of Program Manual Range of use: times, median = 9.25 attitude scales, 1 - 6, with 1= strongly disagree and 6= strongly agree (medians) Helpful to set up: 5.46 Helpful reference: 6.00 Still use it over time: 6.00 Helpful at tailoring: 5.00 their “purple bible”...they used it to help them recruit staff, to keep on track with the community assessment, to keep the goals and objectives in the forefront of their thinking...it has been indispensable for them “the keystone training prevention manual that all prevention manuals should be measured against in the future... and I look at a lot of manuals…” he knows that Joey doesn’t like it because it is overwhelming...Jason tries to get him to look at it more...but he doesn’t

Evaluation of Overview Video Range of use: times, median = 1.00 Helpful to inform Core Group: 5.33 Helpful to understand “look” of program: 5.66 Teach coordinators: 5.55 Still use: 4.04 good to show in the community...short and sweet...gives a good overview and good to show potential members of a core group, providers loves the video... thinks it is so cute...it’s great...he always tears up when they show the graphs about what impact it had...he is craving that so much in his work... he didn’t think that some of the stuff was right for their culture...the African American guy in the film wasn’t like they are...he thinks we could make it better by seeking more information from the inner city...

Evaluation of Training Range: times, median = 0 (but most went to training before baseline) Helpful to talk with others doing MP 6.00 Helped to decide to implement: 5.50 Enjoyed 5.75 Adjusted approach because of training 5.75 training was excellent...it kept his attention for the entire three days it was great to touch base with all the people who were implementing and make those connections... have always wanted a training version 2.0 where the groups addressed issues that they encountered through implementation and adaptation in their communities the training didn’t address organizational issues.... It didn’t encourage you to get organizational buy in, and that’s important to have as well

Evaluation of TA by phone Range: times, median = 2.00 TA receive when call CAPS is helpful 6.00 Helped in obtaining additional funding 4.00 Like getting TA by phone 5.55 Helpful when CAPS initiates calls 5.58 when TA provider called, it was a huge relief on a psychological level to have someone on the phone who had implemented the project before......if someone calls occasionally from the outside, it makes him feel less isolated.... they would like TA provider to visit the program....the guys get a little frustrated because sometimes the information that is provided doesn’t always hit the mark....it would be helpful if the TA provider came out to visit and saw the community and the program

Evaluation of Website Range: times, median = 1.33 Helpful in answering questions about MP 4.00 Help generate ideas for events 5.00 Help generate ideas for outreach 5.00 the web site is nice...as a resource, it is one of the easiest to navigate...it takes you to information quicker, and because it is on line, you can word search it, which is faster [than manual] he downloads the modules from on line and prints them out so he can mark them up (vs. manual) doesn’t really use it...it’s hard to find time to fit it all in gets more out of speaking with TA provider than the web site

Summary of Utilization & Attitudes Strongly positive attitudes towards MPTES But utilization is substantially lower than would be expected by attitudes –possibly because staff is so busy with work that little time to use Biggest issues/dissatisfactions: –Want TA providers to come to CBOs in person –Want more help in adapting intervention –Especially questions/concerns about MP and how to adapt to African American MSM and rural areas

Acknowledgements The translation of research into practice (TRIP) team: John Hamiga, David Sweeney, Scott Tebbetts & Ben Zovod also: Emily Arnold, Michael Foster, Dave Huebner, Wayne Steward and 70+ CBOs trying to prevent HIV/AIDS Funding: NIMH