Research Designs for Complex Community Interventions for Childhood Obesity Prevention Robert W. Jeffery, Ph.D. Division of Epidemiology University of Minnesota School of Public May 4-5, 2004 Robert W. Jeffery, Ph.D. Division of Epidemiology University of Minnesota School of Public May 4-5, 2004

Hypothesis Changes in the food environment of middle schools and high schools will reduce the prevalence of overweight in students.

Design Option: 2-Group Randomized Trial Unit of randomization is school Primary outcome, change in BMI, is measured on students. Two-year duration Unit of randomization is school Primary outcome, change in BMI, is measured on students. Two-year duration

School Selection: Eligibility Criteria Co-educational Student ages At least 2-year attendance expectation Feasibility of intervention Feasibility of data collection Consent of school Satisfactory consent procedure for students Co-educational Student ages At least 2-year attendance expectation Feasibility of intervention Feasibility of data collection Consent of school Satisfactory consent procedure for students

Recruitment: Enrollment Procedure Random selection of schools to approach Possible stratification by public/private, SES, ethnic composition (stratification very limited) Age range (e.g., middle school, high school) Random selection of schools to approach Possible stratification by public/private, SES, ethnic composition (stratification very limited) Age range (e.g., middle school, high school)

Intervention Components: Food Service School lunch A la carte Vending machines School lunch A la carte Vending machines

Food Service Targets Reduce overall availability of food at school Increase availability of low-energy food Decrease availability of high-energy food Increase price of high-energy food Decrease price of low-energy food Reduce food advertising on campus Reduce overall availability of food at school Increase availability of low-energy food Decrease availability of high-energy food Increase price of high-energy food Decrease price of low-energy food Reduce food advertising on campus

Primary Outcome Change in BMI measured at baseline, 12 months, and 24 months Measured in as many students as possible who attend school for the 2-year period Change in BMI measured at baseline, 12 months, and 24 months Measured in as many students as possible who attend school for the 2-year period

Secondary Measures Individual level: (self-report) – Diet – Physical activity – Demographics – Psychosocial measures – Intervention exposure School level: (direct observation) – Intervention implementation – Food sales Individual level: (self-report) – Diet – Physical activity – Demographics – Psychosocial measures – Intervention exposure School level: (direct observation) – Intervention implementation – Food sales

Pretest-Posttest Control Group Design ROXOOR000ROXOOR000 ROXOOR000ROXOOR000

Threats to Validity: Internal Differential experience not related to treatment as a function of treatment group Measurement or dropout bias by treatment group (lack of blinding is problematic, but unavoidable) Differential experience not related to treatment as a function of treatment group Measurement or dropout bias by treatment group (lack of blinding is problematic, but unavoidable)

Threats to Validity: External Volunteer bias, both individual and school Limited school pool Feasibility constraints Overall sample size Volunteer bias, both individual and school Limited school pool Feasibility constraints Overall sample size

Pragmatic Issues: Cost Maintaining good fidelity of intervention delivery can be costly, since school food service, a la carte food service, and vending involve different business operations. Objective assessment of food sales can also be costly. Number of schools is a major determinant of cost. Maintaining good fidelity of intervention delivery can be costly, since school food service, a la carte food service, and vending involve different business operations. Objective assessment of food sales can also be costly. Number of schools is a major determinant of cost.

Pragmatic Issues: Quality Control Political considerations often make intervention implementation incomplete. Food service is a dynamic activity that requires careful monitoring. Large sample size may reduce quality of intervention. Political considerations often make intervention implementation incomplete. Food service is a dynamic activity that requires careful monitoring. Large sample size may reduce quality of intervention.

Analysis Issues: Sources of Variance in Outcomes Individual characteristics Intervention variability School effect Individual characteristics Intervention variability School effect

Analysis Options: By School Maximum protection against threats to internal and external validity Including individual characteristics and intervention fidelity terms reduces variance Cost of producing enough replications to achieve adequate power is problematic Maximum protection against threats to internal and external validity Including individual characteristics and intervention fidelity terms reduces variance Cost of producing enough replications to achieve adequate power is problematic

Analysis Options: By Individual Internal validity threatened by differential factors between schools other than intervention—plausibility of this threat is estimable by historical data Individual level and intervention delivery effects may again be useful covariates External validity and generalizability are weak with this approach Internal validity threatened by differential factors between schools other than intervention—plausibility of this threat is estimable by historical data Individual level and intervention delivery effects may again be useful covariates External validity and generalizability are weak with this approach

Summary & Conclusion The group randomized trial is the gold standard for evaluating community intervention trials. It is strong on both internal and external validity and thus provides a strong evidential basis for important decisions like public health policy. It is not necessary for establishing the plausibility of an intervention approach. Thus, it is best thought of as representing the culmination of a line of research in an area rather than a prerequisite for research funding related to community intervention. The group randomized trial is the gold standard for evaluating community intervention trials. It is strong on both internal and external validity and thus provides a strong evidential basis for important decisions like public health policy. It is not necessary for establishing the plausibility of an intervention approach. Thus, it is best thought of as representing the culmination of a line of research in an area rather than a prerequisite for research funding related to community intervention.