ABCs of Satisfaction & Impact Surveys Elizabeth Harris, Ph.D. Evaluation, Management & Training Associates, Inc. May 21 st, 2008
ABCs of Satisfaction & Impact Surveys Objectives –Understand the importance and relevance of science- based surveying –Understand method of developing telephone and Internet surveys to measure inquirer satisfaction and key outcomes important to contact centers –Leave the workshop with the tools necessary to develop surveys tailored to the needs of each contact center Workshop exercise
Lessons Learned from surveying for: Centers for Disease Control & Prevention –Consumer Response Services Center (CDC-INFO) –Monitor calls & responses First 5 LA –Countywide Parent Helpline
We will discuss: Survey: –Design To Dos Common Pitfalls –Administration Options –Data Use
Satisfaction & Impact Satisfaction: –Immediate –Callers opinion about Overall Service Various Aspects of the Service (Drill Down) Impact: –What happened as a result of the call?
Satisfaction & Impact Surveys: Step 1 What is important to us when we consider the callers perspective and experience? What questions are our key stakeholders asking that we need to answer?
Satisfaction & Impact Surveys: Step 2 Determine method of administration –Interactive Voice Response –Live Surveyor –Web-Based Feasibility Balance between cost & knowledge gain
Satisfaction & Impact Surveys: Step 3 Develop questions Response options –Closed Yes/No Likert Scale –e.g. Very Satisfied to Very Dissatisfied, Press 1 for Very Satisfied –Allow a middle option? Neither Satisfied or Dissatisfied –Open Pro –Information Con –Labor consideration
Satisfaction & Impact Surveys: Step 3 Develop questions Handout example of closed and open- ended questions
Satisfaction & Impact Surveys: Step 4 Design Survey –Question wording Neutral –Response order effect Carefully consider which question is first and which is last
Satisfaction & Impact Surveys: Step 4 Design Survey –IVR Rules Brief and to the point No more than six response options –Consider length Longer = more respondents drop off –Language options
Satisfaction & Impact Surveys: Step 5 Determine Survey Protocol –Who is surveyed-Immediate Satisfaction Every caller –Works well if no automated transfer option TrainingBias Random selection –Works well with a robust system program algorithm that automatically chooses
Satisfaction & Impact Surveys: Step 5 Who is surveyedImpact (Call Back) –Again consider: Purpose of the survey Purpose of the survey –Caller preference –Part of service follow up effort How the information will be used –Other considerations: Labor allocation/resources –To survey –To interpret/make use of information
Satisfaction & Impact Surveys: Step 6 Pilot Testing –Works out the bugs –Improves introductory instructions –May revise questions and responses based on pilot test results
What to Expect: Response Rate Immediate Satisfaction Survey –22% to 33% Follow Up Survey –Will reach half of those you try to call If you try more than once Vary time/day called
Results: Over 90% give the First 5 LA Parent Helpline the highest satisfaction rating for overall service! Over half who received a referral successfully accessed services
Results: Over 75% give CDC-INFO the highest satisfaction rating for overall service Over 75% report learning something new from their call Over half report that what they learned makes them want to change a behavior
Questions?
Exercise: See handouts Select one question you would like to answer with a satisfaction survey
ExerciseBreak Out: Develop a survey question and responses: –Doable/feasible to implement in your setting
ExerciseDebriefing: Share examples from each group
Questions? Contact information: Elizabeth Harris, Ph.D