1. Designing a Study: Critical Evaluation and Measurement

2. Overview Reliability Reliability Validity Validity Demand Characteristics Demand Characteristics Measurement Scales Measurement Scales

3. Reliability  Reliability: the degree to which the same event or measure produces the same result each time it is measured. Any Measure = True Score + Random Error

4. Reliability of Measures Less error

5. Remember… A reliable test is not necessarily valid! A reliable test is not necessarily valid!

6. Threats to Internal Validity Selection/Volunteer bias Selection/Volunteer bias Regression towards the mean Regression towards the mean Repeated testing Repeated testing Attrition/Mortality Attrition/Mortality

7. Construct Validity  Construct validity- Is the measure that is used actually measuring the construct it is intended to measure?

8. Indicators of Construct Validity Face Validity: it looks good. Face Validity: it looks good. Concurrent Validity: it matches current behavior Concurrent Validity: it matches current behavior Predictive Validity: it predicts future behavior Predictive Validity: it predicts future behavior Convergent Validity: it matches other measures Convergent Validity: it matches other measures Discriminant Validity: it doesn’t fit other measures Discriminant Validity: it doesn’t fit other measures

9. Demand Characteristics  extraneous cues that guide a participant’s behavior :  reactivity: A measure is reactive if awareness of being measured changes an individual’s behavior  social desirability: participants may behave in ways they think are more ‘acceptable’

10. How can we reduce demand characteristics? unobtrusive measures unobtrusive measures e.g. one-way mirrors, hidden cameras, prisms in periscopes under the bathroom stall door… e.g. one-way mirrors, hidden cameras, prisms in periscopes under the bathroom stall door… deception deception a cover story to disguise the true purpose a cover story to disguise the true purpose distractor or filler tasks distractor or filler tasks keeps attention away from purpose keeps attention away from purpose

11. Variables and Measurement Scales  Nominal/Categorical Scales  Ordinal Scales  Interval Scales  Ratio Scales

12. Identify the measurement scale for each of the following data: 1.Circle your marital status: Married Single Divorced Engaged 2.Do you go to work today? Yes No 3.If you work, how many hours a week do you work? _______ 4.Rate the extent that you enjoy college on the scale below. 1234512345 Not at all Very much 5.Name your top 3 favorite TV shows in order of preference: 1. ___________ 2. ___________ 3. ____________