Recommendations for a Campus Survey Tool Orlando Leon, Enterprise Architect - Education, ITS Beth Berrean, Communications/Website Services Manager, ISU Opinder Bawa, Chief Technology Officer, ITS

Overview Charge Analysis Current Progress Next Steps

Our Charge Should we purchase and support Qualtrics as an enterprise solution for the Campus? Should UCSF decide on a single solution for all survey needs? Should UCSF rationalize availability and support of existing survey solutions currently within the Campus?

Analysis: Sample Use Cases Encountered Education –Student Learner Survey –Evaluation –Feedback –Registration Research –Radiology Orthopedics Study –mOst Survey Tool –ATHENA –Clinical Trial Qualification Surveys –Research Questionnaires

Analysis: Additional Use Cases Encountered Administrative –Polls –Voting –Usability Satisfaction/Feedback Surveys –Interview Candidate Feedback Clinical –Nursing PI Surveys –Patient Satisfaction

Analysis: Existing Survey Tools

Findings on Tools: Qualtrics Able to collect ePHI SSO integration with MyAccess Mobile versions of surveys “out of the box” Integration with Salesforce objects Supports 48 languages ADA compliance-checking tool Panels Triggers Robust user/group/organization/library features Programmatic interface to administration tools Training and Support

Considerations In conversations with stakeholders (likely survey users—SON, SOM MedEd, Nursing PI, SOM TAC), people asked us to consider: –Product Features –External/Vendor Support –Internal Product Ownership –Integration with Existing Systems –Total Cost of Ownership –Expertise in Survey Design vs Knowledge of the Technology

Recommendations Continue support for RedCap research, Salesforce for research and clinical needs Replace Checkbox with Campus license for Qualtrics Create a strategic marketing plan to help move all qualified users into Qualtrics Create product oversight group for each part of the Campus that intends to use the tool (Schools, Centers, Departments, Administrative) as first line of support for tool

Cost Projection – est $48,000 –$20,000 - Annual license cost –$20, $80k/year for brand administration, organization configuration and custom template creation –$5,000 – 0.05 $100k/year for MyAccess integration –$6,000 – 0.10 $60k/year for marketing and communications est $24,000* –$20,000 - Annual license cost –$4, $80k/year for brand administration * The assumption is that each functional unit using Qualtrics will administer their own users and organization configuration, while the overall brand administrator administers the overall Campus/global configuration and setup of new functional units

Scope of License The term of the license for the Qualtrics Software is one year, beginning January 16, 2012 and includes an unlimited number of surveys. The Qualtrics license may be used by academic areas within University of California San Francisco for projects that are academic in nature. Academic projects may serve research (instructional or academic) and administrative purposes. The license may not be used for purely corporate or "for-profit" purposes. Such corporate projects would include purely clinical and patient related surveys that are not related to academic research. Affiliated hospital and clinics may purchase Qualtrics access according to Qualtrics corporate pricing model. A 10% discount will be applied to any license purchased for corporate purposes. 11

Current Progress BAA reviewed and signed License negotiated, reviewed, and signed –License terms –Features Advisory and Oversight Committee –SON, SOM/UME/GME, SOD, SOP, Graduate Division, Academic Senate, Memory and Aging, Center for Health Professions, CTSI, Library, SAA, ITS Configuration and Deployment –MyAccess/SSO, Wiki/FAQ, Branding, Marketing/Communications, Administration 12

Next Steps Review and Consultation with University Relations and UCSF Legal Finalization of division and general use policies Marketing and Communication to UCSF community Group Training Sessions 13

Findings on Tools: CheckBox (v4) User interface: OK No end user (survey creator) training program As currently implemented at UCSF, –Not able to collect ePHI –No support for multi-lingual surveys –Allows importing of data (Medical Students since 2004) –Integrated with AD (cannot integrate with MyAccess) Key Feature: Actively Used –The SOM MedEd group uses this tool widely and SOM ISU also uses it. It is stable, and there is a upgrade path.

Findings on Tools: RedCap Able to collect ePHI User interface: complex ARS offers classes As currently implemented at UCSF, –No support for multi-lingual surveys –No native accessibility features –Not integrated with MyAccess –Mobile compatible? Key Feature: Validated Instruments –Data collection instruments and forms reviewed for research relevance, accuracy in coding and function by the RedCap Library Oversight Committee.

Findings on Tools: Salesforce - Health Quality Surveys (HQS) Able to collect ePHI Integrated into mobile application development platform Open-source survey definition language supported by Harvard, MIT, and the Boston Children's Hospital Key Feature: Integration with Salesforce –Integrated with multiple systems (including multiple EHR systems) and the UCSF IDR “Anything is possible.”

Findings on Tools: Others SurveyMonkey –ADA compliant –Free Zoomerang SurveyGizmo –Mobile versions built into platform –Professional services available –Integration with Salesforce

Next Steps Endorse recommendation Decide where within the enterprise the product should be managed Negotiate a BAA and license agreements with Qualtrics Create a Product Oversight Committee Develop a marketing/communications plan for helping internal constituents identify survey needs and tools 19