1. Tell me what to ask Designing Surveys: Practice makes pretty good Research & Evaluation Strategic Planning & Implementation Kamehameha Schools Presented by: Brandon Ledward, Wendy Kekahio, Summer Keli’ipio, Nolan Malone and Kathy Tibbetts 10 December 2007

2. Activity 1 Please stand if you’ve ever: (No shame! Get prizes, you know.)

3. Agenda Activity 1 How do we collect data? Activity 2 Before you get started Activity 3 Creating a survey Item Types Activity 4

4. How do we collect data? Quantitative Surveys/evaluations Forms Roll books/ participation counts Assessments/tests Collaborators Qualitative Observations Interviews Artifacts Open-ended survey questions Do you really need/want a survey?

5. Activity 2 Envelope contains 5 scenarios What method of data collection would you use? Why? –Interview –Focus Group –Survey –Observation There’s no wrong answer

6. Before you get started What is a survey? Method of gathering information –Sample vs. Census Every individual is asked the same question in more or less the same way Create a composite profile of a population Types of surveys: –Mail –Telephone –In-person (“paper & pencil” or “show of hands”) –Internet

7. Before you get started How do I create a good survey? The quality of a survey is largely determined by its purpose and the way it is conducted. Scheuren, Fritz. “What is a Survey?” 2004.

8. Before you get started How do I begin? Focus on the purpose of the survey –Where does the purpose come from? See Workshop #1: Logic Models Are Our Friends Know how you will use the information Allocate resources accordingly –Do you need to give makana? –What time of day is best? –Can the survey take a long time?

9. Activity 3 With someone at your table discuss one program outcome you each wish to measure –Why do you need to measure this? –How are you going to use the information? –What are some challenges? Be prepared to share with the group

10. Creating a survey Step 1: Define the purpose Why do you need a survey? What are you going to do with the information? (i.e. reporting) Have you verified the purpose with leadership and staff?

11. Creating a survey Step 2: Plan the administration of the survey What method? (mail, in-person, internet, telephone, etc.) How many people will take the survey? How long will the survey take? When will the survey be administered? What logistical issues should be considered? (age group, materials, makana)

12. Step 3: Write questions & then pilot those questions Step 4: Review & revise your questions and fully implement your survey Creating a survey easier said than done! Checklist Revisions  Approval  START! 

13. Survey Basics: Item Types Two item types: Open-ended Close-ended –Ratings –Multiple choice(s)

14. Survey Basics: Item Types Examples: Open-ended questions 1. What was the single best part of your experience today? ______________________________ 2a. Did participating in this program impact your future career goals? oYes oNo oNot sure 2b. Please explain: ______________________________

15. Survey Basics: Item Types Examples: Close-ended questions  Rating scales 1. Rate each activity: 2. I learned educational strategies I can use in my classroom. 1 2 3 4 5 6 7 8 9 10 Strongly Disagree Strongly Agree

16. Survey Basics: Item Types Examples: Close-ended questions  Multiple Choice(s) 1. What language is primarily spoken in your home? (Choose one only). ○ English ○ Pidgin ○ Hawaiian ○ Some other language (specify):__________

17. Activity 4: Writing your questions Go back to the program outcome that you wanted to measure: Write down 3-5 questions to measure your program outcome. With someone at your table, discuss how you would turn one of these questions into a survey item. Be prepared to share with the group

18. Survey Basics: Reporting Begin at the first step –If you cannot find a reason to report a piece of information don’t waste a question on your survey Know beforehand how info will fit into your report Be brief in reporting all findings

19. Mahalo a nui loa Additional questions? Brandon LedwardSummer Keli‘ipioWendy Kekahio 541-5325534-3964523-6373 brledwar@ksbe.edusukeliip@ksbe.eduwekekahi@ksbe.edu Nolan Malone Kathy Tibbetts 534-3849 541-5321 nomalone@ksbe.edu katibbet@ksbe.edu