Diploma Plus Effective Supports Network Meeting Winter 2009 Questionnaire Design Diploma Plus Effective Supports Network Meeting Winter 2009 Diploma Plus

Questionnaire Design Questionnaire: a set of questions designed to generate the data necessary to accomplish a research project’s objectives Diploma Plus

Questionnaire Design No rules can guarantee a flawless questionnaire Flaws may only be discovered after collection of data Pre-test if possible – small number of key stakeholders (students, parents, staff) Diploma Plus

Questionnaire Design Process Have a clear definition of the research problem or objectives Diploma Plus

Questionnaire Design Process Does each question have the most appropriate form (e.g., structured vs. nonstructured)? Is each question relevant and properly worded to obtain meaningful, valid responses? Diploma Plus

Questionnaire Design Process Is the sequencing of the questions likely to introduce any bias? Are the layout and appearance conducive to accurate and easy data collection (this includes length)? Diploma Plus

Question Relevance and Wording Can the respondent answer the question? How many formative assessments have you administered to your students in the past 6 months? ______ In your opinion, how many of your students are interested in contemporary philosophy? Ten percent ____ Twenty percent ____ Thirty percent ____ Etc… Diploma Plus

Question Relevance and Wording Will the respondent answer the question? If sensitive or embarrassing May refuse May be less truthful May irritate Diploma Plus

Question Relevance and Wording Avoid Double-Barreled Questions Do you feel your teacher’s lessons are relevant and engaging? Yes ____ No ____ No opinion ____ Diploma Plus

Question Relevance and Wording Avoid Leading/Loaded Questions Steer respondents toward a certain answer, regardless of what their true responses are High risk when the focus is on people’s attitudes, beliefs, or opinions Diploma Plus

Question Relevance and Wording Avoid Leading/Loaded Questions “Do you think your teachers this year are worse than your teachers last year?” Diploma Plus

Question Relevance and Wording Avoid One-Sided Questions Presents only one aspect of the issue Do you feel your school is concerned about their teachers? ____ Yes ____ No ____ No opinion Do you feel your school is concerned or unconcerned about their teachers? ____ Concerned ____ Unconcerned Diploma Plus

Question Relevance and Wording Avoid Questions with Implicit Assumptions Why don’t you come to school some days? ___ Can’t wake up ___ Don’t feel like it ___ To avoid a test Why are there some classes you don’t pass? ___ They’re too hard ___ I don’t understand the material ___ Passing is not important to me Diploma Plus

Question Form Nonstructured Questions (Qualitative Data) Permit free responses When responses are expected to be brief Beginning an interview with an open-ended question is worthwhile Help to ensure smooth transition from one major topic to another Diploma Plus

Question Form Structured Questions (Quantitative Data) Two basic forms: Dichotomous questions; two answer choices, typically yes/no format Multiple-category questions; >2 choices Diploma Plus

The Likert Scale For each of the listed statements, please check the one response that best expresses the extent to which you agree or disagree with that statement. Statements Definitely Agree Generally Agree Slightly Agree Slightly Disagree Generally Disagree Definitely Disagree I enjoy my math classes. ___ I feel safe at school. My teachers respect me. This slide, along with the next three draw heavily upon the excellent examples provided by the authors in Exhibits 13.6 through 13.14 of our text. By taking your class participants through each of these question formats with an eye to the exhibits they should begin to become familiar with the wide variety of options available to them in this final stage before they set about fine tuning the survey instruments they’ve developed to date. It’s recommended that you draw upon some of the excellent “hands-on” activities regarding these question formats which surface in the review and discussion questions and “Student Projects”. You can either cover all the formats and then compliment them with exhibits and examples or interject the exhibits and examples as you proceed with each format. Diploma Plus

The Behavior Intention Scale Type of School Event Definitely Would Consider Attending Probably Would Consider Attending Probably Would Not Consider Attending Definitely Would Not Consider Attending Talent show □ Football game Jazz Concert Step show Quiz Bowl Craft fair Student-Teacher basketball game Diploma Plus

A. Graphic Rating Scales Usage (Quantity) Descriptors Use DP.net All the Time Never Use DP.net 10 20 30 40 50 60 70 80 90 100 Smiling Face Descriptors: 1 2 3 4 5 6 7 Before moving into this “value added” territory it’s important to emphasize that balance is the goal, especially when it comes to considering what “belongs” and “what doesn’t” in the questionnaires designed by your students. A crucial distinction for them to take away is between comparative and noncomparative scale formats. Be sure to spend some time on it before going on to cover the five options which conclude this coverage of question formats in Chapter Thirteen. B. Performance Rating Scales Performance Level Descriptors: Truly Terrible Truly Exceptional Poor Fair Average Good Excellent 1 2 3 4 5 6 7 Diploma Plus

Questionnaire Design Sequencing of Questions Personal or demographic at the end May irritate; affect willingness to complete Put at beginning only if they serve as a filter Sensitive questions at the end Simple questions at the beginning Diploma Plus

The Role of a Cover Letter A solid cover letter is set up with the intention of: Increasing a respondent’s willingness to complete the survey and return it in a timely fashion Improving the response rate associated with the overall goals of the research endeavor Introducing the research project and the research team to the respondent Instilling a feeling of trust and respect with the respondent, for the study and the client/research team soliciting the data As noted in the Instructor’s Manual, you need to decide how much emphasis you want to place on the next two slides. Recently, class participants have provided evidence that they’ve been provided with ample information about the role of a cover letter, the guidelines which need to be brought alongside its development and examples. Still, the rewarding thing about this discussion is how it conveys the information (which students may already have via a career development class or student success programs) with an eye to the principles and practice of doing marketing research. Diploma Plus

Guidelines for Developing a Cover Letter Factors Description Personalization Cover letter should be addressed to the specific prospective respondent; use research firm’s professional letterhead stationery. Identification of the organization doing the study Clear identification of the name of the research firm conducting the survey or interview; decide on disguised or undisguised approach of revealing the actual client (or sponsor) of the study. Clear statement of the study’s purpose and importance Describe the general topic of the research and emphasize its importance to the prospective respondent. Anonymity and confidentiality Give assurances that the prospective respondent's name will not be revealed. Explain how the respondent was chosen, and stress that his or her meaningful input is important to the study’s success. Again, this is where it makes some sense to highlight how aspects of cover letter development shift in important and exciting ways when a cover letter is used in a research study. One of the best ways of meeting some of the “we’ve had all this in our career management classes” comments is to note that while Factor #1 is shared by cover letters directed to employers and those designed to introduce and legitimate research studies, it can be argued that Factors #2-#10 are singular to the marketing research industry. Once your class participants recognize this “shift” the justification for proceeding with a more in-depth discussion of the importance of cover letters surfaces nicely. Diploma Plus

Guidelines for Developing a Cover Letter Factors Description General time frame of doing the study Communicate the overall time frame of the survey or interview. Reinforcement of the importance of the respondent’s participation Where appropriate, communicate the importance of prospective respondents’ participation. Acknowledgment of reasons for nonparticipation in survey or interview Point out “lack of leisure time,” “surveys classified as junk mail,” and “forgetting about survey” reasons for not participating, and defuse them. Time requirements and compensations Clearly communicate the approximate time required to complete the survey; discuss incentive program, if any. Again, this is where it makes some sense to highlight how aspects of cover letter development shift in important and exciting ways when a cover letter is used in a research study. One of the best ways of meeting some of the “we’ve had all this in our career management classes” comments is to note that while Factor #1 is shared by cover letters directed to employers and those designed to introduce and legitimate research studies, it can be argued that Factors #2-#10 are singular to the marketing research industry. Once your class participants recognize this “shift” the justification for proceeding with a more in-depth discussion of the importance of cover letters surfaces nicely. Diploma Plus

Guidelines for Developing a Cover Letter Factors Description Completion date and where and how to return the survey Communicate to the prospective respondent all instructions for returning the completed questionnaire. Advance thank-you statement for willingness to participate Thank the prospective respondent for his or her cooperation. Again, this is where it makes some sense to highlight how aspects of cover letter development shift in important and exciting ways when a cover letter is used in a research study. One of the best ways of meeting some of the “we’ve had all this in our career management classes” comments is to note that while Factor #1 is shared by cover letters directed to employers and those designed to introduce and legitimate research studies, it can be argued that Factors #2-#10 are singular to the marketing research industry. Once your class participants recognize this “shift” the justification for proceeding with a more in-depth discussion of the importance of cover letters surfaces nicely. Diploma Plus

CONSUMER BANKING OPINION STUDY BATON ROUGE, LOUISIANA Marketing Resources Group 3205 Windsor Oaks Drive, Suite 1105 Baton Rouge, Louisiana 70814 CONSUMER BANKING OPINION STUDY BATON ROUGE, LOUISIANA If you have a bank account— We need your opinion, With more bank locations, new banking services, and the growth of credit unions and savings and loans, financial institutions are indeed changing. These changes will have an effect on you and your family, and that’s why your opinion is important. Your name has been selected in a sample of Baton Rouge residents to determine what people in our community think about the present products and services offered by banks. Your individual opinions in this survey can never be traced back to you, and all results will be held strictly confidential. The results of the study will provide the banking industry with insight into how to better serve the needs of its customers. The brand-new quarter enclosed with this letter is not enough to compensate you for your time, but it may brighten the day of a youngster you know. Thank you for your assistance. Sincerely, Thomas L. Kirk MRG Project Director P.S. Please return no later than June 21, 2002. A postage-paid envelope enclosed. In an effort to “add spice” to this discussion, it would be useful if you can access any cover letters which might have accompanied research efforts performed by a Department for Institutional Research or a Research Center on campus. Diploma Plus