Questionnaire Construction Assessment Team November 6, 2007 PowerPoint by JD White and Joel H. Scott

When using a Questionnaire… Think about the purpose of your assessment. Then, think about the data you want to collect. Next, ask yourself if a questionnaire is the right tool for the job. Remember, good questionnaires take time to develop. They are not just questions on a page.

Steps for Developing Questionnaires Develop the Purpose Create the Conceptual Framework Write Questions Design the Questionnaire

Developing the Purpose This step likely started with your assessment plan. Think about what, specifically, you want to know, why a questionnaire is appropriate, and what you want to do with the data. Your purpose will assist in keeping you focused while developing your questionnaire.

Creating the Conceptual Framework This is a multi-step process and will include a great deal of brain storming. Step 1: List all the questions/areas you might want to know Step 2: Organize these questions into categories and subcategories Step 3: Define these categories Step 4: Make sure that you are asking items that will provide enough data to fully answer your research assessment questions.

Can you name the Seven Wonders of the World? Did you start to list the Seven Wonders of the World or did you just say “yes” or “no”? The most appropriate answer would be either yes or no. Question wording matters because it will affect the data you will collect.

Drafting Questions This step can be challenging because it asks us to rethink how we ask questions. Tip: Think about why you are asking a question and if you really need the information. Questions have two distinct parts. 1. Question Wording 2. Response Format

Question Wording Be as short and simple as possible.  Ask one question at a time.  Don’t use jargon and the negative wording of a question.  Define/clarify items as needed.

Question Wording cont.  Respondents don’t have the best memories, so be specific and try not to ask about generalities.  Be careful when asking threatening questions.

Response Formats Open response formats don’t limit/influence responses, but they can create data management issues. Use open-ended responses strategically. -Before asking specific questions - As summary for missed information Provide labels where possible Ex. How many days…? ___days

Closed response formats can limit/influence responses, but they provide some control. Response Formats Categories: Use to measure factual information and behavior. Make sure options are exhaustive and mutually exclusive. Ordinal and Interval: Use the items to measure the intensity of attitudes or opinions and think about what scaling you should use (e.g., Likert, Uni-Dimensional, Semantic Differential).

Issues to Consider To Neutral or Not to Neutral. Think about how a respondent enters a response scale and how many decisions the person must make. Strongly Disagree DisagreeNeither Agree nor Disagree AgreeStrongly Agree 12345

Issues to Consider cont. Pre-coding can save time with data entry. Has this presentation increased your understanding of questionnaire development? 1  Yes 2  No There is more than one way to ask a question. Use a variety of question formats. Pre-code

Types of Questions Yes/No: Are you currently a member of the A-Team (Assessment Team)? o Yes o No Uni-dimensional: How interested are you in being a member of the A-Team (Assessment Team) beginning fall 2007? Not At AllModerately Interested Very Interested Semantic Differential: This presentation is: Boring123Interesting

Designing the Questionnaire  Begin with “easy” questions first and more to more “challenging” questions.  Think about placing demographic questions at the end if possible.  Create a conversational tone in the opening text and throughout the questionnaire.  Limit demographic questions to only those that you need.

Designing Questionnaires  Create sections to break up longer questionnaires.  Ask about behaviors before attitudes or opinions.  Use page space effectively.  Design can make a difference.

Formatting the Questionnaire  Margin size, layout, and font size can make a big difference.  Tables are a good tool for helping to format your questionnaire.  Bullets and symbols can provide response circles, boxes, or other options.  Shading can assist the respondent in completing the questionnaire.

What do you think? Were you satisfied with the quality of our services and programs? A. Yes B. No Double Barrel: Ask only one question at a time

What do you think? What brand of television do you own? A. Samsung B. Panasonic Too exclusive: What about other TVs? What about the use of “Other”?

What do you think? In the past two years, approximately how many times have you been to the gym? A times B times C times D. More than 30 times Can you recall things from 2 years ago? What about the past two weeks?

What do you think? Where did you grow up? A. country B. farm C. city Confusing Response Options: What if I live on a farm?

What do you think? Are you in favor of Proposition 13? ____ Yes ____ No ____ Undecided Assumptive Question: Does everyone know what Prop 13 is? Also, what if you “don’t know”?

What do you think? What percent of your annual budget do you spend on advertising for student programs? ______% Too difficult: You don’t want your respondents to pull their hair out.

What do you think? Don’t you think this administration is spending too much money in the Iraq war? A. Yes B. No Leading Question: Be careful of sharing your agenda.

What do you think? What income tax bracket do you report under to the IRS? A. under 30,000 B. between 30,000-99,000 C. 100,000 to 500,000 Threatening question/Be careful of using abbreviated terms-spell them out!

What do you think? 1. Do you currently own a vehicle? If no, go to question 3. A. Yes B. No 2. How many miles do you drive on a daily basis? ______ Be careful of BRANCHING Q’s

Why not this… How many miles do you drive on a daily basis? A. none, I do not own a vehicle B miles C miles D. 21 or more miles E. Other________________miles

What do you think? What do you think about this report? A. It’s the worst report I’ve read B. It’s somewhere between the worst and best C. It’s the best report I’ve read Little variability in responses…How are you going to measure this?

Why not this… Please respond to the degree you agree or disagree with the following statements. I think this report is well written. I think this report should be shared with everyone in the company. I think this report has the potential to improve working conditions.

Questions?