CSC Alumni Survey Harry Bui Jared Guan Ryan Guest

Outline  Background  Goals and Objectives  Procedures  Things That We Kept in Mind  Data Collection Outcome  Changes to Survey  Web Survey  Conclusion

Background  Receive feedback on CSC program  Given to CSC alumni and seniors  About 50 questions, Four pages  20 to 30% response rate

Goals and Objectives  Not to determine better incentives  Evaluate current CSC Alumni Survey  Prevent users from quitting while taking the survey  Incorporate team assignments with project  Redesign, redevelop, and reorganize  Convert paper version to web survey

Procedure  Data Collection  Data Analysis  Application of Results

Procedure: Data Collection  Setup survey stations  Users took survey  Question users for feedback

Procedure: Data Analysis  Read user feedback  Analyzed mouse metrics  Analysis of observations

Procedure: Application of Results  Shortened survey  Removed redundancy  Clarified questions  Restructuring

Things That We Kept in Mind  Information is not lost  Reduce redundant information  Survey shouldn’t be longer  Organize logically  Use the feedback/observation from data gathering lab  Allow participants a way to submit the survey

Data Collection Outcomes Positives  Few leading questions  Very detailed questions  Gathered a lot of data Negatives  “Too long” / “I quit”  “Redundant information”  “Too wordy”

Changes  Moved away from WebSurveyor  Reordered scales to comply with common web practices  Grouped logical information and combined questions  Replaced text areas with radio buttons where appropriate

Survey Comparison  Original Survey Original Survey Original Survey  New Survey New Survey New Survey

Benefits of Web Survey  Web vs. Mail Response rate 20 percent higher than mail Response rate 20 percent higher than mail Overall cost is lower Overall cost is lower  Web vs. Phone 33% (web) vs. 18% (phone) said that they would definitely participate in future studies 33% (web) vs. 18% (phone) said that they would definitely participate in future studies “No Response” answered more frequently in phone surveys “No Response” answered more frequently in phone surveys Conway, Malcom. Web Survey Guidelines

Web Survey Guidelines  Questions are visible on the same screen and not split between screens.  Use page numbers or a progress bar graphic for long surveys.  Limit the amount of information on a screen to minimize scrolling.  Don’t force the respondent to answer a question before the next one is shown.  Allow respondents to interrupt and reenter the survey.  Limit the use of graphic images. Conway, Malcom. Web Survey Guidelines Conway, Malcom. Web Survey Guidelineshttp://

Web Survey  Web Survey Web Survey Web Survey

Conclusion  Discovered a lot problems with old survey  Verified with data collection  Improved usability after data analysis

Questions?