1. Centre for Behavioural Research and Program Evaluation The Centre for Behavioural Research and Program Evaluation is a national program of the National Cancer Institute of Canada funded by the Canadian Cancer Society, and is located at the University of Waterloo. Is Our Scientific Culture an Impediment to the Health of Canadians? Roy Cameron and Barbara Riley CPHA, Halifax June 2, 2008

2. CBRPE Vision and Mission Vision  To transform cancer prevention and supportive care systems to improve the lives of Canadians. Mission  To create knowledge, relationships and resources that contribute to the continuous improvement of population level interventions.

3. Overview: In a nutshell  There is an urgent need to generate relevant evidence to guide population based chronic disease prevention intervention  Scientific organizations and dominant scientific culture are not oriented to support the science that must be done: if we do not support relevant science, are we not impeding the health of Canadians?  Scientific organizations and practices are starting to change in ways that enable generation and use of relevant evidence

4. BC ActNow: The Problem Causes of Premature Mortality and Years Lived in Poor Health Source: Adapted from Evaluation of the Burden of Disease in British Columbia. Strategic Policy and Research Branch, B.C. Ministry of Health, January 2001.

5. BC ActNow: The Business Case Revenue Growth – 3% Education Growth – 3% Health Growth – 8% Balanced Budget Other spending reaches zero by 17/18

6. Conceptual Framework Coordination and Stewardship Aboriginal Relations & Reconciliation Aboriginal Relations & Reconciliation Public Safety & Solicitor General Public Safety & Solicitor General Children & Family Develop. Children & Family Develop. Premier All sectors, municipalities, Non-Governmental Organizations, Agencies, Businesses and Industry Strategic Communications ActNow BC provides a unifying brand for the strategic cross governmental and cross-sectoral initiative for creating a healthy BC population. Facilitates “improved alignment of cross-ministry policy” Economic Development Economic Development Labour & Citizen’s Services Labour & Citizen’s Services.Environment Employment & Income Assistance Employment & Income Assistance Energy, Mines & Petroleum Res Energy, Mines & Petroleum Res Small Bus. & Revenue Small Bus. & Revenue Attorney General Attorney General Committee on Natural Resources & the Economy Committee on Natural Resources & the Economy Committee on Social Development Committee on Social Development StrongStart BC Cabinet Committee Agriculture & Lands Agriculture & Lands Finance Advanced Education Advanced Education Forests & Range Forests & RangeHealth Transportation Community Services Community ServicesEducation Tourism, Sports & Arts Tourism, Sports & Arts Minister of State

7. Some Practical Questions What is the right mix of interventions to implement? What is most urgent and important to do given limited resources and existing assets? What are residents ready for? What will work under conditions (demographic, physical, economic, etc) in this province? What is feasible? How will be know that we’ve made a difference?

8. Answering Practical Questions Traditional approach: Researcher-initiated studies Example: School Health Chronology, University of Waterloo Research Groups  RCTs of smoking prevention programs (including questions about delivery)  Began to doubt generalization through time given how rapidly youth culture evolves  Programming also evolves; ongoing innovation and adaptation by practitioners

9. Answering Practical Questions Alternative approach: Experimentation by social actors. Lesson from tobacco control: “Bal laughs when asked about the role of science in guiding policy decisions… ‘where there is no science you have to go and be venturesome—you can’t use the paucity of science as an excuse to do nothing…all the scientists came in behind us and analyzed what we did’” (Sweet M & Moynihan R. 2007. Improving Population Health: The Uses of Systematic Reviews. Milbank Memorial Fund and Centers for Disease Prevention and Control.)

10. Answering Practical Questions “Bal is frustrated by colleagues who wait for high-level evidence before acting…. ‘Most scientists will say you need a randomized controlled trial level of proof to do a community intervention. That’s horse feathers. We tried twenty-five things—twelve worked and we renewed those. Empirical trial and error is the oldest scientific device and we used it to distinction.’” (Sweet & Moynihan, 2007) Proposition: The most important experiments will not be done by scientists, but by social actors.

11. Big Implications for the Role of Science in Population Intervention Using existing evidence: Bring to bear any relevant evidence in formulating a policy (e.g. principles of behaviour and attitude change to inform tobacco warning labels) Generating new evidence: Studying innovative policies as they are implemented; use evaluation methodologies to study these “natural experiments” (e.g. Charlton 2004. Natural experiment. In Lewis-Beck, Bryman, Liao (eds) The Sage Encyclopedia of Social Science Research Methods (Vol 2). Sage Publications.)

12. Scientific Obligation: CBRPE Position Core obligation: Study the most important questions, in the most rigorous way possible, given the conditions Be willing to trade off rigor for relevance (Green, Glasgow…) Foundational assumption: the scientific community has an obligation to help find solutions to urgent problems that threaten the health and social fabric of Canada; refusing to remain “pure” by doing only studies that use the most rigorous designs is not defensible given that we are funded by taxpayers and donors who depend on us.

13. Scientific Impediments: Examples Structural:  e.g. grant competition timelines (e.g. annual competitions) are impediments to studying natural experiments, which are usually fleeting opportunities Cultural, Normative:  scientist tendency to equate value with rigor, and to base peer judgment on rigor vs. relevance, potential impact  lack of urgency: “some details could be improved, let her reapply next year” – another year is 10% of the elapsed time till BC is broke

14. The Good News: Science is Evolving Example #1: Methods of defining Best Practices Best science (strength of evidence based on internal validity) vs Most impact (promise table: Swinburn, Gill, Kumanyika, Obesity prevention: a proposed framework for translating evidence into action. Obesity reviews, 2005, 6, 23-33.)  Impact Effectiveness LowMediumHigh LowLeast promising Very promisingMost promising MediumLess promising PromisingVery promising HighPromisingLess promisingPromising

15. The Good News: Science is Evolving Example #2: Population Health Intervention Research Initiative for Canada (PHIRIC)  Goal is to increase the quantity, quality and use of population health intervention studies  Catalyst for aligning organizational efforts to enable a ‘new’ science of population intervention  Pan-Canadian  Early days

16. The Centre for Behavioural Research and Program Evaluation is supported by the National Cancer Institute of Canada with funds from the Canadian Cancer Society. We are located at the University of Waterloo. Centre for Behavioural Research and Program Evaluation