MetroHealth Cleveland, OH Presented by: Josh Kratz, MA, Project Coordinator PI: Ann K. Avery, MD Project Manager: Jen McMillen Smith, MSSA, LISW-S Evaluator: Mary M. Step, PhD

Implementation Timeline 7/27/16 – Alpha testing 9/12/16 – Submission to Apple Store & MetroHealth Server 9/29/16 – Ambassador enrollment 9/29/16 – 10/13/16 – Community building 10/13/16 – Begin criteria-eligible enrollment 10/31/16 – First application update 11/30/16 & 12/23/16 - Additional updates expected Enrollment (as of 11/8/16) 9 ambassadors, 11 criteria-eligible users = 20 total (Ambassador badges, private Tales of Triumph, app issue and abuse reporting, and bug-fixes)

Enrollment Target: N = 15 criteria-eligible participants per quarter Current Enrollment (N = 20) Characteristic n Gender Male Female Transgender 14 4 2 Race African American Multi-racial White Hispanic 3 1 Sexual Orientation Gay Heterosexual Bisexual Pansexual 11 Age 18-24 25-29 30+ 5 Target: N = 15 criteria-eligible participants per quarter N = 60 by 8/31/17 N = 100 for 18-mo enrollment period

Implementation process status IRB 7/20/16 – Initial IRB approval 9/23/16 – Most recent revision approval Staffing Fully staffed – New: Project Coordinator, Biostatistician Completed Protocols 1. Enrollment 4. Incoming Communication Flow 2. In-App Admin Posting 5.* Brand Book / Style Guide 3. Social Media Posting CAB Next meeting February 2017 – Content feedback & Tales of Triumph (Summary of revisions: additional visit for evaluation, updated recruitment, updated enforcement/study withdrawal protocol, added new participant handouts)

Implementation Challenges SOLUTION(S) Clear communication of expectations for app functions with tech developer (Blackbird) Negotiating a written list of clear deliverables with timeline Logistics of app (IOS) hosting & server Using app store with password & advocating for tech developer with hospital IT leadership Sketchy wifi connectivity @ MetroHealth Purchased a “hotspot” Literacy, logistic & length challenges with ACASI multisite evaluation Good feedback loop to UCLA, clear need for different voice options Social media surprises: Twitter for pros, IG for PLWHA Used Instagress boost based on specific #’s: ex: TCC2016, HIV, Safe on the Scene Clinic location change within the hospital Literally meeting with participants where they are

Recruitment Recruitment Strategies & Pools In Clinic Other Referrals Community Outreach / Partners Physician Referrals Out of Care: 173 out of 576 were criteria eligible in Nov. Newly Diagnosed: Metro usually receives 100+ newly diagnosed a year. Roughly 60% criteria eligible Community Outreach: This is includes conferences (Transforming Care, etc.), regional meetings (HIV Case Management Meeting), outreach to partner agencies, and events (such as Pride) Other strategies include: word of mouth, and targeting those in care but with high viral loads (no way to quantify this number) Newly Diagnosed (~ 60 per year) Out of Care (EMR list = 173 in Nov. 2016) Social Media

Recruitment Tools & Where We’re At Promotional Materials Social Media Instagram 90 posts 453 followers 1271 likes Twitter 44 followers 6,791 impressions in last 28 days 53 Profile Visits in last 28 days Facebook - 90 likes  - 547 reach in the last 28 days - 2 new page likes in the last 28 days - 7 page views in the last 28 days  - 61 post engagements in the last 28 days  Print Materials

Recruitment Challenges SOLUTION(S) Over scheduling (Due to ACASI) Scheduling fewer enrollments daily No shows / reschedules Texting/Calling all participants on the day of enrollment appointment Hours Being flexible - extended hours Lost to follow-up Additional outreach attempts & saturation of app across social media/clinic/community Saying yes to satisfy doctor (rare) No pressure, accepting disinterest External circumstances in patients’ lives Staying in communication, offering support, & being understanding/patient

Discussion PI: Ann K. Avery, MD | aavery@metrohealth.org Jen McMillen Smith, LISW-S | jmsmith@metrohealth.org Mary M. Step, PhD | mstep@kent.edu Josh Kratz, MA | jkratz@metrohealth.org