QUESTIONNAIRES ORANGE BOOK CHAPTER 9

WHAT DO QUESTIONNAIRES GATHER? BEHAVIOR ATTITUDES/BELIEFS/OPINIONS CHARACTERISTICS (AGE / MARITAL STATUS / EDUCATION LEVEL / SELF-ASSESSMENT) EXPECTATIONS

WHAT ARE SOME USES OF QUESTIONNAIRES ? SCORE SOMEONE ON A SCALE (SUCH AS INDIVIDUALIST / COLLECTIVIST) STUDY INTERACTIONS BETWEEN GENETICS AND ENVIRONMENTAL FACTORS TWIN STUDIES FIELD EXPERIMENTS STUDENT OPINIONS PSYCHOMETRIC TESTS

SOME GENERAL PRINCIPLES ASK THE MINIMUM AMOUNT FOR THE REQUIRED RESEARCH PURPOSE THEIR TIME IS PRECIOUS, DON’T WASTE IT DON’T ANNOY THEM EITHER MAKE SURE THE QUESTIONS CAN BE ANSWERED MAKE SURE THE QUESTION CAN BE ANSWERED TRUTHFULLY MAKE SURE THE QUESTION WILL BE ANSWERED AND NOT REFUSED

FEATURES OF GOOD QUESTIONNAIRES THEY SHOULD DISCRIMINATE AS WIDELY AS POSSIBLE ACROSS THE VARIETY OF HUMAN RESPONSE SHOULD BE HIGHLY RELIABLE SUPPORTED BY TESTS OF VALIDITY

TYPES OF ATTITUDE SCALES EQUAL APPEARING INTERVALS - THURSTONE 1931 AGREE WITH THE STATEMENT, AND A NUMBER OF AGREEMENT TO DERIVE STRENGTH RHS IB SHOULD GIVE MORE WORK (1.2) RHS IB STUDENTS ARE SLEEP-DEPRIVED (9.1) DONE ON A 1-11 SCALE, 11 BEING TOTAL AGREEMENT SUMMATED RATINGS (LIKERT 1931) IMPROVEMENT ON THURSTONE SCALE, PRODUCES A EQUAL NUMBER OF FAVORABLE AND UNFAVORABLE STATEMENTS ABOUT THE ATTITUDE SCALE RHS IB STUDENTS GET ENOUGH SLEEP 1. STRONGLY DISAGREE 2. DISAGREE 3. NEUTRAL 4. AGREE 5. STRONGLY AGREE SEMANTIC DIFFERENTIAL (OSGOOD ET AL 1957)

CLOSED OR OPEN ENDED QUESTIONS? ADVANTAGES OF CLOSED ENDED RESPONSES PARTICIPANT CAN DO IT QUICKLY EASY TO COMPARE PARTICIPANT ANSWERS MORE LIKELY TO GATHER SENSITIVE TOPIC ANSWERS REPLICATION EASIER

CLOSED OR OPEN ENDED QUESTIONS? DISADVANTAGES OF CLOSED-ENDED QUESTIONS RESPONSE SCALES SUGGEST ANSWERS THAT PARTICIPANTS MIGHT NOT HAVE COME UP WITH THEMSELVES PARTICIPANTS CAN RESPOND EVEN IF THEY KNOW NOTHING ABOUT THE TOPIC CLOSED-ENDED QUESTIONS LIMIT PARTICIPANT CHOICES SIMPLE ANSWERS TO COMPLEX QUESTIONS

THINGS TO PREPARE FOR SOCIAL DESIRABILITY PROBLEM – GIVING THE RESEARCHER WHAT THEY WANT USE LIE SCALES (REVERSED ORDER LIKERT SCALE QUESTIONS) SAMPLING USUALLY OPPORTUNITY SAMPLES, FOR AVAILABILITY RELIABILITY EITHER SPLIT-HALF RELIABILITY TEST OR TEST-RETEST RELIABILITY TEST VALIDITY PREDICTIVE VALIDITY IS TESTABLE

THINGS TO AVOID IN STATEMENT CONSTRUCTION COMPLEXITY TECHNICAL TERMS AMBIGUITY DOUBLE-BARRELED TERMS (TWO QUESTIONS AT ONCE) DOUBLE NEGATIVES EMOTIVE LANGUAGE LEADING QUESTIONS INVASION OF PRIVACY BALANCED OF SCALED ITEMS SENSITIVITY OF SCALE ITEMS TO LEVEL OF MEASUREMENT (QUESTION FITS THE LEVEL CHOSEN DON’T DO YES/NO BUT ASK LIKE A LIKERT)

EVALUATING DATA FROM QUESTIONNAIRES? 4 WAYS: EXPERIMENT DV RESPONSE FREQUENCIES CENTRAL TENDENCY (MEAN, MEDIAN, MODE) & DISPERSION (INTERQUARTILE RANGE, STANDARD DEVIATION) CORRELATIONS

INTERPRETING CORRELATIONS POSITIVE, NEGATIVE OR NEITHER? NEVER CAUSAL!!! MAKE A SCATTERPLOT, FIND THE CORRELATION COEFFICIENT (WEAK , MODERATE , STRONG )

HOW CAN WE INCREASE THE VALIDITY OF CORRELATION STUDIES? MULTIPLE REGRESSION STUDIES PARTIAL REGRESSION STUDIES PATH ANALYSIS

ADVANTAGES OF CORRELATION STUDIES INEXPENSIVE AND EASY TO GATHER DATA CONFIRM OTHER RESEARCH IN THE TRIANGULATION PROCESS POINT TO FUTURE HYPOTHESES USEFUL WHERE ETHICS PRECLUDE EXPERIMENTATION, OR EXPERIMENTATION NOT PHYSICALLY POSSIBLE CAN RELATE QUESTIONS TO OTHER DATA

ADVANTAGES AND DISADVANTAGES OF QUESTIONNAIRES ADVANTAGES LESS INSTRUMENTATION ERRORS MINIMAL EXPERIMENTATION BIAS ANONYMITY COST-CONSCIOUS DATA COLLECTION DISADVANTAGES SOCIAL DESIRABILITY PROBLEMS POOR QUESTIONS INCREASE INSTRUMENTATION ERRORS ETHICS IF FLAWED QUESTIONS OR NOT JUSTIFIED MEMORY & EXPERIENCES BIAS ANSWERS MEASURE ONLY PERCEPTIONS