1. Recommendations for a Campus Survey Tool Orlando Leon, Enterprise Architect - Education, ITS Beth Berrean, Communications/Website Services Manager, ISU Opinder Bawa, Chief Technology Officer, ITS

2. Overview Charge Analysis Recommendations Next Steps

3. Our Charge Should we purchase and support Qualtrics as an enterprise solution for the Campus? Should UCSF decide on a single solution for all survey needs? Should UCSF rationalize availability and support of existing survey solutions currently within the Campus?

4. Analysis: Sample Use Cases Encountered Education –Student Learner Survey –Evaluation –Feedback –Registration Research –Radiology Orthopedics Study –mOst Survey Tool –ATHENA –Clinical Trial Qualification Surveys –Research Questionnaires

5. Analysis: Additional Use Cases Encountered Administrative –Polls –Voting –Usability Satisfaction/Feedback Surveys –Interview Candidate Feedback Clinical –Nursing PI Surveys –Patient Satisfaction

6. Analysis: Existing Survey Tools

7. Findings on Tools: CheckBox (v4) User interface: OK No end user (survey creator) training program As currently implemented at UCSF, –Not able to collect ePHI –No support for multi-lingual surveys –Allows importing of data (Medical Students since 2004) –Integrated with AD (cannot integrate with MyAccess) Key Feature: Actively Used –The SOM MedEd group uses this tool widely and SOM ISU also uses it. It is stable, and there is a upgrade path.

8. Findings on Tools: RedCap Able to collect ePHI User interface: complex ARS offers classes As currently implemented at UCSF, –No support for multi-lingual surveys –No native accessibility features –Not integrated with MyAccess –Mobile compatible? Key Feature: Validated Instruments –Data collection instruments and forms reviewed for research relevance, accuracy in coding and function by the RedCap Library Oversight Committee.

9. Findings on Tools: Salesforce - Health Quality Surveys (HQS) Able to collect ePHI Integrated into mobile application development platform Open-source survey definition language supported by Harvard, MIT, and the Boston Children's Hospital Key Feature: Integration with Salesforce –Integrated with multiple systems (including multiple EHR systems) and the UCSF IDR “Anything is possible.”

10. Findings on Tools: Qualtrics Able to collect ePHI (after signing a BAA) SSO integration with MyAccess Mobile versions of surveys “out of the box” Integration with Salesforce objects Supports 48 languages ADA compliance-checking tool Panels Triggers Robust user/group/organization/library features Programmatic interface to administration tools Training and Support

11. Findings on Tools: Others SurveyMonkey –ADA compliant –Free Zoomerang SurveyGizmo –Mobile versions built into platform –Professional services available –Integration with Salesforce

12. Considerations In conversations with stakeholders (likely survey users—SON, SOM MedEd, Nursing PI, SOM TAC people asked us to consider: –Product Features –External/Vendor Support –Internal Product Ownership –Integration with Existing Systems –Total Cost of Ownership –Expertise in Survey Design vs Knowledge of the Technology

13. Recommendations Continue support for RedCap research, Salesforce for research and clinical needs Replace Checkbox with Campus license for Qualtrics Create a strategic marketing plan to help move all qualified users into Qualtrics Create product oversight group for each part of the Campus that intends to use the tool (Schools, Centers, Departments, Administrative) as first line of support for tool

14. Cost Projection 2011-2012 – est $48,000 –$20,000 - Annual license cost –$20,000 - 0.25 FTE @ $80k/year for brand administration, organization configuration and custom template creation –$5,000 – 0.05 FTE @ $100k/year for MyAccess integration –$6,000 – 0.10 FTE @ $60k/year for marketing and communications 2012-2013+ est $24,000* –$20,000 - Annual license cost –$4,000 - 0.05 FTE @ $80k/year for brand administration * The assumption is that each functional unit using Qualtrics will administer their own users and organization configuration, while the overall brand administrator administers the overall Campus/global configuration and setup of new functional units

15. Next Steps Endorse recommendation Decide where within the enterprise the product should be managed Negotiate a BAA and license agreements with Qualtrics Create a Product Oversight Committee Develop a marketing/communications plan for helping internal constituents identify survey needs and tools 15