1. H676 Week 5 - Plan for Today Review your project and coding to date
Complex Data (TEXT chapters 22-26)
Multiple independent subgroups
Multiple outcomes or time-points
More than one comparison or treatment group
Review your project and coding to date
Statistical Power - Jaehun
TEXT chapter 29 and Valentine et al., 2010
Complex Interventions (Multiple Readings)
Map aspects of complexity onto sources of evidence

2. Complex Data (Borenstein et al., 2009)
Multiple independent subgroups within studies
E.g., by age, gender, ethnicity, etc.
Multiple outcomes or time-points within studies
E.g., Knowledge, attitudes, behavior, health indicators
E.g., immediate, 3-months, 12-months
More than one comparison or treatment group
Multiple interventions
By dosage or type
Multiple controls –
E.g., treatment as usual, placebo, no treatment
All require separate rows when coding

3. Potential Moderators Variation between studies in
Year of study or intervention delivery, published or not
Research design
Types of intervention or control conditions
Sample characteristics
E.g., demographics
Assessed outcomes and measures of each outcome
Multiple outcomes, Measures or Times
All variations are coded in separate columns
E.g., one column to indicate level of each variable for each row

4. Review of Coding 1: First columns
First columns include:
Study label and
Statistics needed to calculate ES
For intervention studies, use separate blocks of columns for:
control and intervention groups
at both pretest and follow-up
Within each study, use one line per moderator:
E.g., condition, subgroup, outcome, time of measure, etc.
See next slide

5. Review of Coding 2: Subsequent columns
Subsequent columns include:
Items that describe the study
Authors, Year
Items that describe the research
Research design, conditions, sample sizes by condition
Items that describe the sample
Age, ethnicity, health status, etc.
For intervention studies, items that describe the intervention and its implementation/delivery
Length (hours days or months), length and # of sessions
Measures of implementation integrity
Any of these factors (except authors) could be considered as moderators

6. Your turn Up to 5 minutes for each of you
Tell us what your project is about
Tell us how far along you are (or are not)
Tell or show us your coding set-up
Do you have any problems that you need help with?

7. Statistical Power
Jaehun

8. Complex Interventions & Evaluations
Complexity in
causes + contexts + interventions
+ implementation & outcomes
 Complex interventions
& Complex research factors

9. Complex Interventions
Multiple components
Multiple groups or organizational levels
Flexibility or tailoring permitted
Self-organization, evolution over time

10. Complex Causal Pathways
Nonlinear relationships, phase changes
Multiple mediators
Multiple moderators
Feedback loops
Synergy between components
Multiple outcomes

11. Complex research questions (Pettigrew et al., 2013)
Start with a straight-forward question
Then consider multiple outcomes
Whether from simple or complex interventions
Some of these link in causal chains
Suggesting mediation
Then consider intervention complexity
Develop a logic model of how the class of interventions being reviewed might work
Code only those intervention components that vary across studies
They become possible moderators
Then consider aspects of context
They also become possible moderators
At all levels, code only those aspects that vary across studies

12. Research Questions about Complex Interventions (Squires et al., 2013)
Research questions should consider PICO(C/S)
Participants, Intervention(s), Comparison(s), Outcomes, and Context or Study design
Complex interventions can contain a mix of effective and ineffective (or even harmful) components
Interactions might be Synergistic (effect is > sum of components) or
Dysynergistic (effect is < sum of components)
Interdependencies might be Contemporaneous (effect of one depends on the presence of another) or
Temporal (effect of one depends on a prior result)

13. Scope of Review: Lumping vs Splitting
Splitters minimize complexity of review by limiting studies to close replications
Lumpers include interventions with different combinations of complexity
A lumped review could end up being “split” into two subgroup analyses
as a result of identified heterogeneity

14. Specification of Complex Interventions
Pragmatic features are easier to define than theoretical differences
Clear definition of prototypical components
Those deemed necessary to define the intervention
Define discretionary components/delivery
Might or might not be present in each instance (code it)
A scoping review can help identify these variations
Logic models help reviewers/readers visualize intervention
Can include implementation, process & economic data

15. Intervention components that can lead to hetero-geneity of results

16. Methodological Characteristics that contribute to heterogeneity (Pigott & Shepperd, 2013)

17. Need for linguistic/conceptual precision (Noyes et al., 2013)
Complexity in:
Intervention
Implementation
Context
Participant responses
Interactions between any two or more of these