Class 9 Dependent Variables, Instructions/Literature Review Class 9 Dependent Variables, Instructions/Literature Review Chapters 13 Spring 2016

Dependent Variables: Continuous Psychometric Properties ◦ Validity ◦ Reliability 2

Validity Construct Validity ◦ Factor structure-latent constructs ◦ Convergent and Discriminate Validity  Correlation with similar/dis-similar measures Predictive Validity ◦ Correlation or Differences across groups ◦ Longitudinal 3

Reliability The extent to which scores show true variance in attributes across participants as opposed to error variance ◦ Assumption: error variance is random, therefore it does not correlate with anything ◦ More items of good quality = higher reliability ◦ One-item scales have very low reliability; estimated around r =.25 4

Types of Measures Observational Measures Self-Report Paper-and-Pencil Measures ◦ Content tests- right/wrong answers ◦ Likert-scales (3 to 7 response options) 5

Sources of Measurement Error Specific error ◦ something unique about the instrument (or observational task) that differs from what the researcher intended (e.g. social desirability; reading level; idioms) Transient error ◦ some temporary factor that affects the measurement (e.g. order of instruments; historical events; noise while observations occur; tiredness; inattention) 6

Sources of Measurement Error Scoring/Recording error ◦ Tend to be random (self-report/raters) Inter-rater disagreement/observer drift ◦ in observation studies, what initially may appear as antisocial behavior may later be judged more normative 7

Types of Reliability Inter-Scorer Test/Re-test Alternate Forms Internal Consistency ◦ how items correlate with each other 8

Internal Consistency Reliability Split-Half Kuder-Richardson ◦ Dichotomous items- right /wrong Chronbach Alpha ◦ Average correlation of all possible split-half reliabilities 9

Well-being Measure: Q 2a-c Well-being Measure: Q 2a-c a. 12-item measure r x =.70 to r x =.95 among college students b. Expected reliability estimates among the adolescent sample c. Three-item measure: r x =.40 to r x =.55 10

Well-being Measure: Q 2a Well-being Measure: Q 2a a. 12-item measure: internal consistency reliability coefficients r x =.70 to r x =.95 among college students  70% to 95% of variability in respondents’ scores is due to and the rest is due to. 11

Reliability Estimates: Q 2a Extent to which variability is due to true variation versus error ◦ Cronbach alpha =. 70;  70% of variation in scores is due to true differences in parenting skills & 30% is due to error ◦ Cronbach alpha =. 95;  95% of variation in scores is due to true differences in parenting skills & 5% is due to error 12

Well-being Measure: Q 2b Well-being Measure: Q 2b a. 12-item measure r x =.70 to r x =.95 among college-age samples b. Expected reliability estimates among the adolescent sample 13

Wellbeing Measure: Q 2b Wellbeing Measure: Q 2b a. 12-item measure r x =.70 to r x =.95 among college samples b. Expected reliability coefficient among adolescents =>.70 Yes/No ? Reliability refers to scores with specific and. It’s a property of not of instruments. 14

Wellbeing Measure: Q 2b Wellbeing Measure: Q 2b a. 12-item measure r x =.70 to r x =.95 among College students b. Expected reliability coefficient among adolescents =>.70 No. Reliability refers to scores with specific respondents and conditions It’s a property of scores, not of instruments 15

Wellbeing Measure: Q 2c Wellbeing Measure: Q 2c c. Three-item measure: r x =.40 to r x =.45. To improve the scores’ reliability just add 5 or 6 items: True False Not sure 16

Wellbeing Measure: Q 2c Wellbeing Measure: Q 2c c. Three-item measure: r x =.40 to r x =.45 – New items increase reliability only if they are of good quality 17

Reliability and Correlation: Q 5 In correlational research, how does the reliability of two scores affect the probability that the observed correlation coefficient between the two scores approximates the “true” correlation coefficient of the constructs? 18

Internal Reliability and Correlation Parenting Skills 1 PSM1 r x =.45 Parenting Skills 2 PSM2 r x = 90 Self-Esteem PSM1 (r x =.45) correl. Self Esteem PSM2 (r x =.90) correl. Self Esteem 19

Reliability vs. Validity Observed correlation coefficient will be smaller and less accurate with the less reliable measure Correlations between constructs are attenuated by the (internal) reliability of the measures The reliability of a measure puts a ceiling on its validity 20

Instruments 1.Description of Measure2.Validity Estimates a.Instrument namea.Convergent/Discriminate Validity b.Acronymb. Validation Sample(s) c.Authors d.Key References 3.Reliability Estimates e.Brief description of construct(s) a.Chronbach’s alpha coefficient f.Type of measure (e.g self-report) Previous and Current studies g.Number of items b. Test/Re-Test h.Example of items i.Items response options j.Factors or subscales k.Scoring options and direction 21