LECTURE 3 EPSY 642 FALL 2009

CODING STUDIES Dependent variable(s) Construct(s) represented Measure name and related characteristics Effect size and associated calculations Independent variables Population Sample Design Potential Mediators and Moderators Bias mechanisms and threats to validity

CODING STUDIES- Dependent Variables Construct name(s): eg. Receptive or Expressive Vocabulary Measurement name: Willson EV Test Raw score summary data (mean, SD for each group or summary statistics and standard errors for dep. var): Exp Mean= 22 Exp SD = 5 n=100, Con Mean = 19 Con SD = 4, n=100 Effect size (mean difference or correlation) e = (22-19)/  (20.5) Effect size transformation used (if any) for mean differences: t-test transform ( e = t (1/n1 + 1/n2) ½ ), F-statistic transform (F) ½ = t for df = 1, 198 probability transform to t-statistic: t(198) =  [probt(.02)] point-biserial transform to t-statistic, regression coefficient t-statistic Effect size transformations used (if any) for correlations: t-statistic to correlation: r 2 = t 2 / (t 2 + df) Regression coefficient t-statistic to correlation

CODING STUDIES- Independent variables Population(s): what is the intended population, what characterizes it? Gender? Ethnicity? Age? Physical characteristics, Social characteristics, Psychosocial characteristics? Cognitive characteristics? Sample: population characteristics in Exp, Control samples eg. % female, % African-American, % Hispanic, mean IQ, median SES, etc.

CODING STUDIES- Independent variables Design (mean difference studies): 1. Random assignment, quasi-experimental, or nonrandom groups 2. Treatment conditions: treatment variables of importance (eg. duration, intensity, massed or distributed etc.); control conditions same 3. Treatment givers: experience and background characteristics: teachers, aides, parents 4. Environmental conditions (eg. classroom, after- school location, library)

CODING STUDIES- Independent variables Design (mean difference studies) 5. Time characteristics (when during the year, year of occurrence) 6. Internal validity threats: maturation, testing, instrumentation, regression, history, selection

CODING STUDIES- Independent variables Mediators and Moderators Mediators are indirect effects that explain part or all of the relationship between hypothesized treatment and effect: (T)e M In meta analysis we establish that the effect of T on the outcome is nonzero, then if M is significantly related to the effect e. We do not routinely test if T predicts M

CODING STUDIES- Independent variables Mediators and Moderators Moderators are variable for which the relationship changes from one moderator value to the next: (T)e for M=1 (T)e for M=2 In meta analysis we establish that the effect of T on the outcome is nonzero, then if M is significantly related to the effect e. We do not routinely test if T predicts M.7.3

Coding Studies- Bias Mechanisms Researcher potential bias- membership in publishing cohort/group Researcher orientation- theoretical stance or background Type of publication: Refereed vs. book chapter vs. dissertation vs. project report: do not assume refereed articles are necessarily superior in design or analysis- Mary Lee Smith’s study of gender bias in psychotherapy indicated publication bias against mixed gender research showing no effects by refereed journals with lower quality designs than non- refereed works Year of publication- have changing definitions affected effects? Eg. Science interest vs. attitude- terms used interchangeably in 1940s-1950s; shift to attitude in 1960s Journal of publication- do certain journals only accept particular methods, approaches, theoretical stances?