Towards Deeper RNS User Engagement: How to get users to sign in for personalized content VIVO 2016 UCSF Clinical and Translational Science Institute (CTSI) 11/16/2018 [ADD PRESENTATION TITLE: INSERT TAB > HEADER & FOOTER > NOTES AND HANDOUTS]

Recap of last year’s presentation Vanity emails and Origin of Dashboard Feature In the past 3 years we’ve had real success sending users emails with year-end summaries/stats Feedback was that: People really liked knowing how ‘popular’ their page is People wanted more, and more specific info about viewers NSF results emailed to researchers Two ‘test’ emails one more general, second named companies (look at slide from 2014 UCCSC) UCSF Clinical & Translational Science Institute 11/16/2018 [ADD PRESENTATION TITLE: INSERT TAB > HEADER & FOOTER > NOTES AND HANDOUTS]

That got us thinking…. http://www.pestrevenge.com/different-types-of-mouse-traps-explained/ Could we build a better mouse trap? Would people sign in to see their stats? Presentation Title and/or Sub Brand Name Here 11/16/2018 11/16/2018 [ADD PRESENTATION TITLE: INSERT TAB > HEADER & FOOTER > NOTES AND HANDOUTS]

The Personalized Dashboard What makes it tick? Google Analytics&APIs Limitations ORNG framework Fancy GUI Design Need more techie description of feature? UCSF Clinical & Translational Science Institute 11/16/2018 [ADD PRESENTATION TITLE: INSERT TAB > HEADER & FOOTER > NOTES AND HANDOUTS]

Launch and Promotion Soft launch in Aug 2015 Email is our primary way of reaching the masses New Features email – Nov 2015 Sent to all profile page owners Personal Statistics email – May 2016 Sent to all profile holders with ‘good’ traffic 4590 – new features email to researchers 3105 - people, for 2+ entities and 10+ visits UCSF Clinical & Translational Science Institute 11/16/2018 [ADD PRESENTATION TITLE: INSERT TAB > HEADER & FOOTER > NOTES AND HANDOUTS]

New Features Email UCSF Clinical & Translational Science Institute 11/16/2018

Email Results Tracking the communications efforts Email statistics were great Comments were also great Our takeaways 2 unsubscribe requests from New Features Email ZERO unsubscribes in May UCSF Clinical & Translational Science Institute 11/16/2018 [ADD PRESENTATION TITLE: INSERT TAB > HEADER & FOOTER > NOTES AND HANDOUTS]

New Sign in Text UCSF Clinical & Translational Science Institute 11/16/2018

What happened to site traffic and sign ins? Photo credit https://static.pexels.com/photos/2120/city-traffic-people-smartphone.jpg UCSF Clinical & Translational Science Institute 11/16/2018 11/16/2018 [ADD PRESENTATION TITLE: INSERT TAB > HEADER & FOOTER > NOTES AND HANDOUTS]

Site Results Did the dashboard feature influence traffic behavior? We looked at users who accessed Profiles from the UCSF network in the last 3 months. Cohort 1: people who did not visit dashboard Cohort 2: people who did 245,554 visits in the three months 245,554 sessions in the past 3 months UCSF Clinical & Translational Science Institute 11/16/2018 [ADD PRESENTATION TITLE: INSERT TAB > HEADER & FOOTER > NOTES AND HANDOUTS]

Visits/Month UCSF dashboard ‘users’ visit 3.7x more often This is just the past three months, so we’re past the bump we got from the May promotional email. Users who access Profiles from on-campus, but not the dashboard, accessed the site an average of 0.8 times a month. And people who accessed the dashboard at least once from on campus, accessed the site 2.97 times a month UCSF dashboard ‘users’ visit 3.7x more often UCSF Clinical & Translational Science Institute

Pages/Visit UCSF Clinical & Translational Science Institute 11/16/2018

Minutes/Visit UCSF Clinical & Translational Science Institute 11/16/2018

correlation ≠ causation Of course correlation isn’t causation UCSF Clinical & Translational Science Institute

Sign Ins Big spikes when we send successful emails. We are seeing a gradual increase over time too. UCSF Clinical & Translational Science Institute 11/16/2018 11/16/2018 [ADD PRESENTATION TITLE: INSERT TAB > HEADER & FOOTER > NOTES AND HANDOUTS]

Tracking Sign Ins Average sign ins per week 50% increase! Show sign in text on profiles pages UCSF Clinical & Translational Science Institute 11/16/2018 11/16/2018 [ADD PRESENTATION TITLE: INSERT TAB > HEADER & FOOTER > NOTES AND HANDOUTS]

Sign Ins 11 people signed in > 100 times in 20 months. Name Anna Orr, PhD 394 Sarah Woolf-King, PhD, MPH 386 Tomasz Nowakowski, PhD 356 Brian Turner 227 Daniel Le Grange, PhD 193 Courtney McFall 183 Amin Khalifeh Soltani, MD 181 Christian Hoppmann, PhD 174 Anthony Shum, MD 154 Michael Marvin 152 Monica McLemore, PhD, MPH, RN 137 Zeanid Noor 127 Eric Meeks 110 Cathra Halabi, MD 103 Il Jin Kim 102 11 people signed in > 100 times in 20 months. UCSF Clinical & Translational Science Institute 11/16/2018 11/16/2018 [ADD PRESENTATION TITLE: INSERT TAB > HEADER & FOOTER > NOTES AND HANDOUTS]

Who are these gung-ho users Who are these gung-ho users? Are they accessing the dashboard more because they’re already frequent users? Or are they frequent users because they’re accessing the site to check their stats? We don’t know — and that’s what we’re hoping/planning to learn next by surveying the top users in recent months. [Public domain / CC0 photo from https://www.pexels.com/photo/healthy-person- woman-sport-4078/]

Take Home Points Promotion + Compelling Content = Deeper User Engagement More sign ins – 50% increase over a year! More and longer visits from UCSF people More pages per visit UCSF Clinical & Translational Science Institute 11/16/2018

Discussion of Future Options What do you think would draw users to sign in/engage? Peer-to-peer messaging? Funding opportunities related to publications/co-authors? Other personal statistics? Regardless, if you launch any new features, promote them! You’ll definitely get a spike in traffic and probably an increase in visit frequency. UCSF Clinical & Translational Science Institute

Thank you! Brian Turner brian.turner@ucsf.edu profiles.ucsf.edu/brian.turner UCSF Clinical & Translational Science Institute