Sample Power No reading, class notes only Last part of this lecture: Black pp 380-385

Sample Power Probability that a sample of given size will allow us to correctly reject the null hypothesis. Null Hypothesis is true: No difference Null Hypothesis is false: There is difference Wrong Decision Null Hypothesis is accepted Correct Decision Type II error 95% probability 20% probability Wrong Decision Null Hypothesis is rejected Correct decision 5% probability Sample Power Type I error 80% probability

Summary Power is the probability of correctly rejecting the null hypothesis Power is related to the mean of the population that the sample is actually coming from Power is related to the SEM (SD & sample size) larger the sample size, greater the power Power is related to the chosen level of significance (usually p=.05)

Example sample size calculation I have two groups that I wish to compare. The mean of group 1 is 50. The mean of group 2 is 55. SD of both groups is 10. What should be my sample size so that I can conclude that the difference is statistically significant? Probability of correctly detecting this difference should be high (usually 80%) SOLUTION: http://www.stat.uiowa.edu/~rlenth/Power/

Power Analysis Example Exercise 1: I wish to estimate the behavior problem scores of children age 12-14 who had school disciplinary problems. This mean is estimated to be 115. I wish to be able to say that this sample of children could not have been generated by a normative population (mean = 100, SD = 15). What should my sample size be?

A useful concept for Power Analysis: Effect Size A standardized way to measure the effects of one variable on another X  Y If X changed by 1 SD, by how many SDs would Y change? How strong is this effect?

Using effect size: Exercise 2 Hypothesis: Children who experienced highly authoritarian parenting are expected to be less empathetic in their dating relationships than children who did not experience high levels of authoritarian parenting because …. What should be your sample size???? What is the IV and what is the DV? What would be your expectation about the size of the effect? Small, medium, or large? How does that translate to the difference between two groups?

Effect Size 0.2 or less: Small effect 0.3-0.4: Moderate effect 0.5-0.6: Large effect 0.7 or greater: Very large effect (almost impossible in Social Sciences)

Power Analysis Example Exercise 3: I wish to compare the mean behavior problems scores of children age 12-14 who have intact or disrupted families. I expect the score in disrupted families to be higher and to me, this score should be at least 0.3 SD higher than the intact families to be meaningful. What should my sample size be?

Power Analysis Example Exercise 4 I would like to test if students with high self esteem give more autonomous decisions in choosing their colleges than students with low self esteem. I expect that the difference between low and high self esteem groups will be small (0.2 SD). What should my sample size be, so that I can detect this small difference with statistical significance at p=0.05 level with 85% power?

Power Analysis Example Exercise 5 I would like to estimate the effects of authoritarian parenting on the level of religious prejudice of adolescents. I expect that this correlation will be about 0.2. What should my sample size be, so that I can detect this modest correlation with statistical significance at p=0.05 level with 85% power?

Power Analysis FINAL Example Exercise 6 [YOUR PARAGRAPH ON SAMPLE POWER] This research is about the effects of interparental violence on the level of intimacy in the dating relationships of college students. It is expected that there is a moderate negative correlation between the level of overt interparental conflict and ability of the college students to maintain intimacy in their dating relationships. Previous studies found this correlation to be around 0.25 (Smith, 1999; Miller, 2001). What should the sample size be, so that this correlation can be detected with statistical significance at p=0.05 level with 80% power?

MEASUREMENT Black 188-199, 201-207

Principles of Measurement Types of variables – Level of measurement: Nominal, ordinal, interval/ratio measurement Types of instruments: Factual data instruments Attitudinal instruments Observational instruments Tests

Validity Validity as a set of interrelated attributes of measurement Construct validity Criterion validity Predictive validity Content validity Face validity

Construct Validity Definition: The degree of consistency between the construct, and its operational definition. How do we measure it? Use theoretical, conceptual, logical arguments Example: Relationship satisfaction Definition of the construct Operational definition

Criterion Validity Definition: The suitability of a measurement instrument for classifying individuals based on a trait. How do we evaluate criterion validity? Compare the scores of individuals who are known to differ on the trait (according to an independent source). Example: Depressive affect scale Independent classification of subjects into depressed and not-depressed groups (e.g., by a clinician)

Predictive validity Definition: The suitability of a measurement instrument for predicting outcomes. How do we evaluate predictive validity? Compare the scores of individuals to their future scores or future performance. Example: OSS Obtain the freshman GPA’s of students

Content Validity Definition: The adequacy of a measurement instrument for representing the construct in its entirety. How do we evaluate content validity? Theoretical consideration of all dimensions of the construct, and verification that all dimensions are considered by the operationalization. Example: Relationship satisfaction What aspects of the relationship?

Face Validity Definition: The perception of the subjects that the instrument is a measurement of the construct of interest. How do we measure it? Talk-through sessions with subjects. Example: Behavior problems Aggressive behaviors Bullying Destroying toys Example: Relationship satisfaction

Reliability Definition: Consistency between two measurements Measure applied twice in two occasions Two halves of the same instrument Measure administered by two different assessors

Reliability For any given measure, there are two sources of variability True variability Variability due to measurement problems or the situation Variance of the “true” score Variance of the “error” in measurement Variance of the total score

Factors affecting reliability and validity Enough questions to elicit needed information Quality of wording Time needed / time allowed to respond Enough heterogeneity in subjects