H676 Week 5 - Plan for Today Review your project and coding to date Complex Data (TEXT chapters 22-26) Multiple independent subgroups Multiple outcomes or time-points More than one comparison or treatment group Review your project and coding to date Statistical Power - Jaehun TEXT chapter 29 and Valentine et al., 2010 Complex Interventions (Multiple Readings) Map aspects of complexity onto sources of evidence

Complex Data (Borenstein et al., 2009) Multiple independent subgroups within studies E.g., by age, gender, ethnicity, etc. Multiple outcomes or time-points within studies E.g., Knowledge, attitudes, behavior, health indicators E.g., immediate, 3-months, 12-months More than one comparison or treatment group Multiple interventions By dosage or type Multiple controls – E.g., treatment as usual, placebo, no treatment All require separate rows when coding

Potential Moderators Variation between studies in Year of study or intervention delivery, published or not Research design Types of intervention or control conditions Sample characteristics E.g., demographics Assessed outcomes and measures of each outcome Multiple outcomes, Measures or Times All variations are coded in separate columns E.g., one column to indicate level of each variable for each row

Review of Coding 1: First columns First columns include: Study label and Statistics needed to calculate ES For intervention studies, use separate blocks of columns for: control and intervention groups at both pretest and follow-up Within each study, use one line per moderator: E.g., condition, subgroup, outcome, time of measure, etc. See next slide

Review of Coding 2: Subsequent columns Subsequent columns include: Items that describe the study Authors, Year Items that describe the research Research design, conditions, sample sizes by condition Items that describe the sample Age, ethnicity, health status, etc. For intervention studies, items that describe the intervention and its implementation/delivery Length (hours days or months), length and # of sessions Measures of implementation integrity Any of these factors (except authors) could be considered as moderators

Your turn Up to 5 minutes for each of you Tell us what your project is about Tell us how far along you are (or are not) Tell or show us your coding set-up Do you have any problems that you need help with?

Statistical Power Jaehun

Complex Interventions & Evaluations Complexity in causes + contexts + interventions + implementation & outcomes  Complex interventions & Complex research factors

Complex Interventions Multiple components Multiple groups or organizational levels Flexibility or tailoring permitted Self-organization, evolution over time

Complex Causal Pathways Nonlinear relationships, phase changes Multiple mediators Multiple moderators Feedback loops Synergy between components Multiple outcomes

Complex research questions (Pettigrew et al., 2013) Start with a straight-forward question Then consider multiple outcomes Whether from simple or complex interventions Some of these link in causal chains Suggesting mediation Then consider intervention complexity Develop a logic model of how the class of interventions being reviewed might work Code only those intervention components that vary across studies They become possible moderators Then consider aspects of context They also become possible moderators At all levels, code only those aspects that vary across studies

Research Questions about Complex Interventions (Squires et al., 2013) Research questions should consider PICO(C/S) Participants, Intervention(s), Comparison(s), Outcomes, and Context or Study design Complex interventions can contain a mix of effective and ineffective (or even harmful) components Interactions might be Synergistic (effect is > sum of components) or Dysynergistic (effect is < sum of components) Interdependencies might be Contemporaneous (effect of one depends on the presence of another) or Temporal (effect of one depends on a prior result)

Scope of Review: Lumping vs Splitting Splitters minimize complexity of review by limiting studies to close replications Lumpers include interventions with different combinations of complexity A lumped review could end up being “split” into two subgroup analyses as a result of identified heterogeneity

Specification of Complex Interventions Pragmatic features are easier to define than theoretical differences Clear definition of prototypical components Those deemed necessary to define the intervention Define discretionary components/delivery Might or might not be present in each instance (code it) A scoping review can help identify these variations Logic models help reviewers/readers visualize intervention Can include implementation, process & economic data

Intervention components that can lead to hetero-geneity of results

Methodological Characteristics that contribute to heterogeneity (Pigott & Shepperd, 2013)

Need for linguistic/conceptual precision (Noyes et al., 2013) Complexity in: Intervention Implementation Context Participant responses Interactions between any two or more of these