1. A New Reporting Guideline for Trials of Social and Psychological Interventions: CONSORT-SPI ESRC Bath 2016 Prof Paul Montgomery- Oxford Dr Jane Dennis- Bristol Project funded by the UK Economic and Social Research Council

2. UK What Works Network -Create, share, and use high-quality evidence on policy programs and practices which combined receive public spending of more than £200 billion -Public health, social care, education, crime reduction, and economic growth US Social and Behavioral Science Team -Assists federal agencies in applying behavioural science insights to policies and operations -Improve public welfare, programme outcomes, and cost effectiveness Applied Behavioral/Social Scientists Live in Exciting Times

3. Obama’s Executive Order in Sept 2015 https://www.whitehouse.gov/the-press-office/2015/09/15/executive-order-using-behavioral-science-insights-better-serve-american

4. We have problems reproducing psychological science (Open Science Collaboration, 2015) -Replications of 100 experimental and correlational studies -97% of original studies vs 36% of replications had significant results Mean effect size was half the magnitude of the mean effect size of the original effects “There is still more work to do to verify whether we know what we think we know” Most Published Research May be False

5. A research finding is less likely to be true (Ioannidis 2005): -when the studies are smaller -when effect sizes are smaller -when there is a greater number and lesser pre-selection of tested relationships -where there is greater flexibility in designs, definitions, outcomes, and analytical modes - when there is greater financial and other interest and prejudice -when more teams are involved in chase of statistical significance Why Most Published Research Findings are False

6. Several stages of research production may lead to waste (Moher 2015) 85% of Biomedical Research Funding ($210 Billion) Is Being Avoidably Wasted

7. How to Make More Published Research True Some research practices that may help increase the proportion of true research findings (Ioannidis 2014): -Large-scale collaborative research -Adoption of replication culture and reproducibility practices -Registration (studies, protocols, analysis codes, datasets, raw data) -Sharing (data, protocols, materials, software, and other tools) -Containment of conflicted sponsors and authors -More appropriate statistical methods -Standardisation of definitions and analyses -More stringent thresholds for claiming discoveries or ‘‘successes’’ -Improvement of study design standards -Improvements in peer review, reporting, and dissemination of research -Better training of scientific workforce in methods and statistical literacy

8. The Lancet REWARD (REduce research Waste And Reward Diligence) Campaign Center for Open Science to improve openness and integrity of scientific practices -Open Science Framework for transparent, cloud-based management of scientific projects -Transparency and Openness Promotion (TOP) Guidelines (Nosek 2015) Berkeley Initiative for Transparency in the Social Sciences (BITSS) Data Access and Research Transparency (DA-RT) Statement for social scientists Meta-Research Innovation Center at Stanford (METRICS) Laura and John Arnold Foundation (LJAF) Research Integrity Grants (over $80 million since 2012) Enhancing the Quality and Transparency of Health Research (EQUATOR) Network reporting guidelines Research Transparency Can Increase Value, Reduce Waste

9. Social and psychological intervention RCTs Reporting Guidelines & CONSORT Developing CONSORT-SPI The CONSORT-SPI Checklist Objectives

10. Mayo-Wilson et al. (2013). Developing a reporting guideline for social and psychological intervention trials. Trials, 14, 242. Grant et al. (2013). Reporting quality of social and psychological intervention trials: a systematic review of reporting guidelines and trial publications. PLoS One, 8(5), e65442 Montgomery et al. (2013). Protocol for CONSORT-SPI: An Extension for Social and Psychological Interventions. Implementation Science, 8, 99. Project Publications

11. Grant et al. (2012). Development of a CONSORT extension for interventions in public health and related disciplines. The Lancet, 380(Supp. 3). S14. Spreckelsen et al. (2012). Letters: Additional requirements for complex interventions. BMJ, 345, e8003. Grant et al. (2013). Letter to the Editor: New guidelines are needed to improve the reporting of trials in addiction sciences. Addiction, 108, 1687-1688. Gardner et al. (2013). Editorial Perspective: New guidelines are needed to improve the reporting of trials in child and adolescent mental health. Journal of Child Psychology and Psychiatry, 54(7), 810-812. Editorials

12. Social/Psychological mechanisms For various health/social issues Behavioural/psychological/structural techniques Naturalistic, “hard to control” settings Social and Psychological Interventions

13. Multiple Intervention Components Behaviours of providers/recipients Various “levels” of intervention/outcome Flexibility/tailoring intervention Complex Interventions: MRC Framework

14. Intensive intervention for chronic juvenile offenders Therapists, caseworkers, psychologists, psychiatrists Work with individual, family, peers, and neighbourhood Settings: home, school, community Services may focus on cognition and behaviour change, communication skills, parenting skills, family relations, peer relations, school performance, or social networks Tailored to the specific needs of the youth and family Example: Multisystemic Therapy

15. Overall intervention effectiveness Key intervention components Target populations/recipients/settings Important implementation factors Systematic Reviews of RCTs

16. Participant and setting characteristics Interventions and their implementation Outcome assessment Theories informing the study Trial design Good RCT Reporting Includes…

17. The Problem: Poor Reporting

18. Minimum set of items on article content Reflect issues related to bias Based on evidence and consensus Reporting Guidelines

20. Our case: The CONSORT Statement

21. CONSORT Extensions

22. Official CONSORT Extension Rigorous consensus development Multi-pronged dissemination strategy CONSORT-SPI Project Phase 1: Lit Reviews Phase 2: Delphi Process Phase 3: Consensus Meeting Phase 4: Write-up Phase 5: Disseminate and Implement

23. Paul Montgomery, University of Oxford Evan Mayo-Wilson, Johns Hopkins University Sean Grant, University of Oxford Geraldine Macdonald, Queen’s University Belfast Sally Hopewell, University of Oxford Susan Michie, University College London David Moher, Ottawa Health Research Institute Project Executive

24. J Lawrence Aber Chris Bonell David Clark Frances Gardner Steve Hollon Jim McCambridge Laurence Moore International Advisory Group Mark Petticrew Steve Pilling Lawrence Sherman James Thomas Elizabeth Waters David Weisburd Jo Yaffe

25. 1. Examine previous guidance What published standards already exist? -E.g. WIDER; JARS; Non-Pharm. Is current guidance sufficiently rigorous? Phase 1: Do we need a new guideline?

26. 2. Assess current reporting quality of social and psychological intervention RCTs - Is reporting adequate across disciplines? - Single set of standards that can apply across disciplines? Phase 1: Do we need a new guideline?

27. Systematic literature search - Multiple electronic databases - Expert databases of guidelines Assessed according to recommended techniques for guideline development (Moher 2010) 1: Review of Previous Guidelines

28. Preliminary Activities - Literature Review Document Development - Consensus Methods (Delphi Exercise, Consensus Meeting) Publication Strategy - Extension and Elaboration Document Dissemination - Journal endorsement, Open Access What makes a good guideline? (Moher et al. 2010)

29. Average percentage of recommended techniques used to develop reporting guidelines Guideline Development Stage CONSORT Documents (n = 6) Non-CONSORT Medical Documents (n = 6) Social Science Documents (n = 7) Preliminary Activities91.7%70.8%67.9% Document Development75.0%44.4%31.0% Publication Strategy66.7%5.5%23.8% Dissemination76.7%10.0%37.1% Median Citations per year (Range) 73.7 (43.3 – 535.5) 9.9 (0.2 – 480.2) 4.4 (1.0 – 65.0)

30. 89 new/modified reporting items not in CONSORT No current guidance sufficiently incorporates all key standards Social/behavioural science guidelines could be more rigorously developed and disseminated Social/behavioural science researchers have not participated in previous CONSORT guidelines Findings

31. Systematic literature search - Multiple electronic databases - Expert databases of guidelines Assessed according to recommended techniques for guideline development (Moher 2010) 2: Review of Reporting Quality

32. Reporting items from guideline review Divided into 3 domains - Internal validity - External validity - Study details Number of RCTs varied per journal - 11 had one RCT to 35 (JCCP) & 39 (C&E) 2: Review of Reporting Quality

33. Average compliance with reporting items per study Reporting AreaClinical PsychologyCriminologyEducation Social Work All Disciplines Internal Validity 47.0%27.4%30.0%41.2%37.6% External Validity 48.4%42.2%47.7%41.8%46.8% Study Details 43.2%20.3%27.2%38.3%33.9% All Standards 47.2%34.6%39.5%41.1%42.2% RCTs per discipline: Clinical Psychology—99, Criminology—31, Education—89, Social Work—20 Number of reporting standards: Internal Validity—40, External Validity—83, Study Details—24

34. Cochrane Risk of Bias Reporting AreaClinical PsychologyCriminologyEducation Social Work All Disciplines Randomisation Sequence 30.1%11.3%18.0%28.8%23.0% Allocation Concealment 26.3%17.2%3.4%28.3%16.7% Blinding 20.2%4.3%11.2%18.3%14.6% Participant Flow 55.4%14.5%20.4%37.5%35.6% Data Analysis 50.0%31.8%36.0%44.6%41.9% Outcomes 67.2%54.8%53.7%56.3%59.6% Protocols 29.9%11.6%14.6%27.0%21.6% RCTs per discipline: Clinical Psychology—99, Criminology—31, Education—89, Social Work—20 RS = 4 Items, AC = 3 Items, B = 3 Items, PF = 8 Items, DA = 13 Items, O = 4 Items, P = 5 Items

35. Implementation Data Reporting AreaClinical PsychologyCriminologyEducation Social Work All Disciplines Intervention: Design 74.1%69.7%79.7%80.0%76.1% Intervention: Delivery 43.8%35.5%44.8%37.9%42.6% Intervention: Uptake 27.8%17.1%26.5%20.7%25.3% Control: Design 60.5%62.1%70.9%43.1%63.1% Control: Delivery 32.3%31.5%41.4%16.2%34.2% Control: Uptake 21.0%18.8%25.8%5.8%21.3% I-Des = 10 Items, I-Del = 12 Items, I-Up = 7 Items, C-Des = 8 Items, C-Del = 12 Items, C-UP = 6 Items

36. Average compliance of RCTs with key reporting standards Ref: Grant et al (2013). PLoS One, 8(5), e65442

37. Social/behavioural science guidelines developed/disseminated with less rigour 89 new/modified reporting standards compared to CONSORT guidelines 239 RCTs report <50% of standards on average Phase 1: Lit Reviews Ref: Grant et al (2013). PLoS One, 8(5), e65442

38. Phase 2: Delphi Process

39. N = 384 (32 countries total) 58 items recommended for inclusion - All but 1 of CONSORT 2010 checklist items (registration) Substantive qualitative feedback for consensus meeting and E&E Phase 2: Delphi Process

40. Possible checklist items were generated from reporting standards found in systematic review (Grant 2013) - CONSORT Statement and relevant Extensions - Guidelines from behavioural/social sciences IAG reviewed draft of Delphi Round 1 and requested changes prior to Delphi launch Items for Round 1

41. PICOT PICOT Theory of Change Theory of Change Secular Events Secular Events Service Environment Service Environment Intervention: Design Intervention: Design Intervention: Delivery Intervention: Delivery Intervention: Uptake Intervention: Uptake Outcome Measures Outcome Measures Data Collection Data Collection Data Analysis Data Analysis 77 Items for Delphi Round 1 Blinding/Masking Blinding/Masking Recruitment Recruitment Baseline Data Results Baseline Data Results Implementation Implementation Generalisability Generalisability Implications Implications Other papers about this trial Other papers about this trial Conflicts of Interest Conflicts of Interest Ethical Approval Ethical Approval Intervention Development Intervention Development

42.  Various SPI stakeholders - Trialists and systematic reviewers - Methodologists - Editors & peer-reviewers - Funding Representatives - Policy- Makers - Practitioners Participants for Delphi Process

43.  Inclusion criteria - Researchers: publish at least one SPI publication - Journal editor: editorial board of journal publishes SPI RCTs - Practitioner: Provide SPI services - Funder: Current position involves funding SPI RCTs - Policy-maker: SPI related public or advisory post - Consumer Rep: current position/membership with organisation Phase 2: Delphi Round 1

44. Recruitment - Journal editorial boards - Intervention literature - Signed up on website - Research society/org - IAG Recommendation - Conference contact - Delphi participant recommendation Phase 2: Delphi Round 1

45.  Procedures - Emailed invitations (and reminders) with survey link - *Survey active for ~4 weeks  *Government shut-down - Ranked items on 1-10 Likert scale - Free text comment box for each item - Analysed and categorised according to median rank, dispersion of scores, and free-text comments - Items produced for Round 2 Phase 2: Delphi Round 1

46. Procedures - Emailed Round 1 participants with link to survey - Survey active for ~10 weeks (due to winter break) - Ranked Round 1 “Indeterminate” items as “Include”, “Exclude”, or “Unsure” - Free text comment box for each manuscript section - Analysed and categorised according to percentage rankings and free- text comments - Items produced for consensus meeting Phase 2: Delphi Round 2

47. N = 384 in Round 1 (32 countries total) - N = 321 in Round 2 (no sig differences) Gender: 48% Female, 51% Male Age: 14% 65 Stakeholder roles: 92% Academic/Researcher, 29% Practitioner, 34% Journal Editor, 12% Funder, 9% Policy-maker, 6% Consumer Rep Phase 2: Delphi Process

48. Delphi Process Participants

49. Delphi Process: Academics/Researchers Academic/ResearcherN% of total sample Trialist17846% Systematic Reviewer21155% Statistician4512% Methodologist15540% Other7520% Total academics/researchers (% of sample) 35592%

50. Items - 77 items in Round 1 Included (n = 36) Indeterminate (n = 41 in Round 1; split to n = 61 for Round 2) New (n = 5) - 66 items in Round 2 Included (n = 22), indeterminate (n = 44) - 58 items recommended for inclusion in CONSORT-SPI Median > 8 and low dispersion in Round 1; or >80% ranking of “Inclusion in Round 2” Phase 2: Delphi Process

51. 31 participants from Delphi Process Structured, face-to-face meeting to discuss and vote on items for CONSORT-SPI Checklist Phase 3: Consensus Meeting Phase 1: Lit Reviews Phase 2: Delphi Process Phase 3: Consensus Meeting Phase 4: Write-up Phase 5: Disseminate and Promote Uptake

52. Consensus Meeting Participants Doug Altman Kamaldeep Bhui Andrew Booth Peter Craig Manuel Eisner Mark Fraser Larry Hedges Robert Kaplan Peter Kaufmann Spyros Konstantopoulos Kenneth McLeroy Brian Mittman Arthur Nezu Edmund Sonuga-Barke Gary VandenBos Robert West

53. Phase 3: Consensus Meeting

54. Introduction to reporting guidelines Review of reporting guidelines and trial reporting quality Group discussion: trials in “your” area(s) Specific issues for SPI trials CONSORT-SPI Delphi process Group discussion: thoughts on Delphi Consensus Meeting: Day 1 Agenda

55. Discuss and vote on items for CONSORT-SPI Checklist - Introduction section - Methods - Results - Discussion section - Other important information Consensus Meeting: Day 2 Agenda

56. - Ample time to allow sufficient time for thorough discussion Want to prevent hasty decision-making - Structured discussions of items proposed for the checklist from the Delphi process - Care taken to ensure: All views are expressed All ideas are considered - Voting is confidential using anonymous ballots to promote honest answers and allow participants to rethink their position if a re-vote is needed Consensus Meeting: Day 2 Agenda

57. What is in CONSORT, is in CONSORT Making decisions about items for CONSORT-SPI checklist Some material will go into E&E guidance (i.e., the “users' manual”) Looking to provide minimum standards and concise items from the entire process - Delphi items longer than desired to make sure Delphi participants were clear on concepts Some materials may be future work building from this meeting! Options on the Table

58. Intervention theory of change Eligibility criteria for settings and providers Intervention/comparator delivery and uptake Intervention materials (e.g., manual, website) How missing data were handled Number approached, screened, and eligible New/Adapted Items

59. Socioeconomic baseline variables Availability of trial data Other potential interests than funder Involvement of the intervention developer Other stakeholder involvement Incentives offered as part of the trial New/Adapted Items

60. Draft official guideline extension Tailored E&E documents to disciplines - Rationale for each item - Examples of good reporting Phase 4: Write-Up

61. Simultaneous co-publication Journal endorsement and adherence Presentations at conferences/meetings Editorials and newsletters Training and education Phase 5: Dissemination

62. Title and abstract CONSORT-SPI Checklist Item #Standard CONSORT DescriptionExtension for CONSORT-SPI 1aIdentification as a randomised trial in the title § 1b Structured summary of trial design, methods, results, and conclusions (for specific guidance see CONSORT for abstracts) § § indicates that an extension item for cluster trials exists

63. Introduction: Background and Objectives CONSORT-SPI Checklist Item #Standard CONSORT DescriptionExtension for CONSORT-SPI 2a Scientific background and explanation of rationale § 2bSpecific objectives or hypotheses § If pre-specified, how the intervention was hypothesised to work

64. Methods: Trial Design CONSORT-SPI Checklist Item #Standard CONSORT DescriptionExtension for CONSORT-SPI 3a Description of trial design (such as parallel, factorial) including allocation ratio § If the unit of random assignment is not the individual, please refer to CONSORT for Cluster Randomised Trials 3b Important changes to methods after trial commencement (such as eligibility criteria), with reasons

65. CONSORT-SPI Checklist Methods: Participants Item #Standard CONSORT DescriptionExtension for CONSORT-SPI 4aEligibility criteria for participants § When applicable, eligibility criteria for settings and those delivering the interventions 4b Settings and locations where the data were collected

66. CONSORT-SPI Checklist Methods: Interventions Item #Standard CONSORT DescriptionExtension for CONSORT-SPI 5 The interventions for each group with sufficient details to allow replication, including how and when they were actually administered § Extent to which interventions were delivered and taken up as planned, including what they actually involved *Where other informational materials about delivering the intervention can be accessed When applicable, how intervention providers were assigned to each group * Indicates item may move to another section

67. CONSORT-SPI Checklist Methods: Outcomes Item #Standard CONSORT DescriptionExtension for CONSORT-SPI 6a Completely defined pre-specified primary and secondary outcome measures, including how and when they were assessed § 6b Any changes to trial outcomes after the trial commenced, with reasons

68. CONSORT-SPI Checklist Methods: Sample Size Item #Standard CONSORT DescriptionExtension for CONSORT-SPI 7aHow sample size was determined § 7b When applicable, explanation of any interim analyses and stopping guidelines

69. CONSORT-SPI Checklist Methods: Randomisation Item #Standard CONSORT DescriptionExtension for CONSORT-SPI 8a Method used to generate the random allocation sequence 8b Type of randomisation; details of any restriction (such as blocking and block size) § 9 Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned § 10 Where applicable, who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions §

70. CONSORT-SPI Checklist Methods: Awareness of Assignment Item #Standard CONSORT DescriptionExtension for CONSORT-SPI 11a Who was aware after assignment to interventions (for example, participants, providers, those assessing outcomes), and how any masking was done 11b If relevant, description of the similarity of interventions

71. CONSORT-SPI Checklist Methods: Awareness of Assignment Item #Standard CONSORT DescriptionExtension for CONSORT-SPI 12a Statistical methods used to compare groups for primary and secondary outcomes § How missing data were handled (e.g., complete case analysis, simple imputation, multiple imputation), with details of any imputation method 12b Methods for additional analyses, such as subgroup analyses and adjusted analyses

72. CONSORT-SPI Checklist Results: Participant Flow Item #Standard CONSORT DescriptionExtension for CONSORT-SPI 13a For each group, the numbers randomly assigned, received intended treatment, and analysed for the primary outcome § Where possible, the number approached, screened, and eligible prior to random assignment, with reasons for dropout 13b For each group, losses and exclusions after randomization, together with reasons §

73. Approached (n= ) Screened/assessed for eligibility (n= ) Excluded (n= )  Not meeting inclusion criteria (n= )  Declined to participate (n= )  Other reasons (n= ) Analysed (n= )  Excluded from analysis (give reasons) (n= ) Lost to follow-up (give reasons) (n= ) Discontinued intervention (give reasons) (n= ) Allocated to intervention (n= )  Received allocated intervention (n= )  Did not receive allocated intervention (give reasons) (n= ) Providers/organisations/areas (n= ) Number of participants by provider/organisation/area (median =... [IQR, min, max]) Lost to follow-up (give reasons) (n= ) Discontinued intervention (give reasons) (n= ) Allocated to intervention (n= )  Received allocated intervention (n= )  Did not receive allocated intervention (give reasons) (n= ) Providers/organisations/areas (n= ) Number of participants by provider/organisation/area (median =... [IQR, min, max]) Analysed (n= )  Excluded from analysis (give reasons) (n= ) Allocation Analysis Follow-Up Randomised (n= ) Enrolment

74. CONSORT-SPI Checklist Results: Recruitment Item #Standard CONSORT DescriptionExtension for CONSORT-SPI 14a Dates defining the periods of recruitment and follow-up 14bWhy the trial ended or was stopped

75. CONSORT-SPI Checklist Results: Baseline data and numbers Item #Standard CONSORT DescriptionExtension for CONSORT-SPI 15 A table showing baseline characteristics for each group § Including socioeconomic variables where applicable 16 For each group, number included in each analysis and whether the analysis was by original assigned groups §

76. CONSORT-SPI Checklist Results: Outcomes and Estimation Item #Standard CONSORT DescriptionExtension for CONSORT-SPI 17a For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% confidence interval) § *Indicate availability of trial data 17b For binary outcomes, presentation of both absolute and relative effect sizes is recommended 18 Results of any other analyses performed, including subgroup analyses and adjusted analyses, distinguishing pre-specified from exploratory 19 All important harms or unintended effects in each group (for specific guidance see CONSORT for harms)

77. CONSORT-SPI Checklist Discussion Item #Standard CONSORT DescriptionExtension for CONSORT-SPI 20 Trial limitations, addressing sources of potential bias, imprecision, and, if relevant, multiplicity of analyses 21 Generalisability (external validity, applicability) of the trial findings § 22 Interpretation consistent with results, balancing benefits and harms, and considering other relevant evidence

78. CONSORT-SPI Checklist Important Information Item #Standard CONSORT DescriptionExtension for CONSORT-SPI 23Registration number and name of trial registry 24 Where the full trial protocol can be accessed, if available 25 Sources of funding and other support, role of funders Declaration of any other potential interests

79. CONSORT-SPI Checklist Stakeholder Involvement Item #Standard CONSORT DescriptionExtension for CONSORT-SPI New Item *Any involvement of the intervention developer in the design, conduct, analysis, and reporting of the trial *Other stakeholder involvement in trial design, conduct, and/or analyses *Incentives offered as part of the trial

80. Applying CONSORT-SPI See Example Paper

81. Article Title - A Pilot Randomised Control Trial: Testing a Transitional Care Model for Acute Psychiatric Conditions 1a: Identification of Randomised Trial in Title

82. Abstract - Objective - Method - Results - Conclusions 1b: Structured Summary

83. Background 2a: Scientific Background & Rationale

84. The article reports findings from research that examined the feasibility and effectiveness of the Naylor TCM for individuals with SMI and comorbid health conditions with the aim of reducing hospital readmissions, reducing emergency department (ED) use, improving continuity of care, and improving health quality of life following a hospitalisation for an acute psychiatric condition. 2b: Specific Objectives or Hypothesis

85. Such as parallel, factorial, including allocation ratio 3a: Description of Trial Design

86. Such as eligibility criteria – with reasons None noted 3b: Important Changes to Methods After Trial Commencement

87. Hospital patients were eligible if they: a)were aged 18 to 64 b)had a diagnosis of an SMI such as schizophrenia, bipolar & major depression c)had a diagnosis of major medical condition such as diabetes, asthma, or cancer d)were English speaking e)resided in the city of Philadelphia 4a: Eligibility Criteria

88. From an inpatient unit of a 515 bed acute care general hospital in a large northeastern urban city. Baseline in hospital and follow up in community 4b: Settings & Locations Where Data Collected

89. Intervention Procedure & Protocol described Usual care meant that a case manager was assigned & psychiatrist provided medication management 5: Interventions for Each Group

90. Completely specified - Health related quality of life - Continuity of care - Service utilisation No changes in outcomes 6a: Outcomes

91. Power not done as pilot study Sample size of 40 No interim analyses 7a & b: Sample Size

92. Sequence generation A computerised randomisation schedule was based on a double parallel method using a sample size of 60 (assuming a single group size of 30 with attrition to 20 participants) with block sizes of 6 participants. This means that for every block 3 were assigned to the intervention and 3 assigned to control. There were a total of 10 blocks in this design. This design chosen to keep the psychiatric NP caseload manageable as we only had one nursing providing intervention. 8: Randomisation

93. Once experimental participants were consented by the research assistant the NP was notified of the patient participant 9: Allocation Concealment Mechanism

94. Randomisation done by computer Research Assistants enrolled participants Research assistants then contacted PI for next allocation assignment 10: Implementation

95. NP aware of experimental intervention participants Research Assistants – not blinded as often need assistance in locating participants for follow up interviews Noted that experimental intervention also received usual care 11: Awareness of Assignment

96. Statistical methods to compare groups Chi square or t test of differences between groups conducted 12: Awareness of Assignment

97. CONSORT flow chart included Losses & exclusions after randomization all noted in flow chart Reasons for losses noted 13: Participant Flow

98. Dates – March 2011 to August 2011 Follow ups – at 6 & 12 weeks from baseline 14: Recruitment

99. Table 2 – characteristics of participants by groups Table 3 health related quality of life by group at baseline (and at 12 week follow up) 15: Baseline Data

100. Participants analysed by assigned group - Analysed outcomes of intervention group = 18 and control = 17 (baseline both groups 20) 16: Numbers Analysed

101. Each outcome for each group – Tables 3, 4, & 5 Effect sizes not noted – outcomes not statistically significant – no difference from zero – except one in wrong direction 17: Outcomes and Estimation

102. Other analyses conducted, e.g., subgroup - none conducted 18: Ancillary Analyses

103. No harms noted 19: Harms

104. Last paragraph indicates limitations – titled limitations 20: Limitations

105. Applicability of trial findings – noted in conclusion of article 21: Generalisability

106. Discussion section 22: Interpretation

107. None noted – not registered 23: Registration

108. Where protocol could be accessed - Not noted - Basically articles covers protocol of study - Referenced to another article from the same study 24: Protocol

109. End of article - Declaration of conflicting Interests – Authors declared no potential conflicts of interest with respect to research, authorship, and/or publication of this article. Funding – from Robert Wood Johnson Foundation Interdisciplinary Nursing Quality Research Initiative 25: Declaration of Interests

110. A New Evidence Grading System for Complex Interventions GRADE-CI https://www.spi.ox.ac.uk/research/details/grade-extension-for-complex-social-inter.html

111. Dr Erik von Elm – Institut Universitaire de Médecine Sociale et Preventive (IUMSP), Lausanne, Switzerland Dr Eva Rehfuess – Institute of Medical Informatics, Biometry and Epidemiology Ludwig-Maximilians – University, Munich, Germany Prof Geraldine Macdonald – University of Bristol, Bristol, UK Dr Jane Dennis – Research Synthesis Ltd, Bristol, UK Prof Paul Montgomery – Centre for Evidence-Based Social Intervention, University of Oxford Dr Sean Grant – RAND Corporation, Santa Monica, USA Dr Susan Norris – Guideline Review Committee Secretariat, WHO Project Executive

112. International Steering Committee Gordon Guyatt Holger Schunemann Peter Tugwell Ian Shemilt Stephanie Chang Andrew Booth Philip Davies Birte Snilstveit Matthew Morton Mark Petticrew Steven Hollon Bonnie Spring Frances Gardner Julia Littell James Thomas Sandra Wilson Manual Eisner

113. The GRADE approach offers a transparent and structured process for developing and presenting (effectiveness) evidence summaries for systematic reviews and for carrying out steps involved in developing recommendations: It specifies an approach to: Framing questions for systematic reviews and guidelines Choosing outcomes of interest and rating their importance Assessing and rating the quality of a body of evidence Incorporating effectiveness evidence with other considerations to arrive at recommendations (DECIDE) The GRADE Approach

114. The GRADE Methodology and Process Question formulation (PICO) Search & retrieval of relevant studies Evidence synthesis Rating the overall quality of evidence Rating the quality of evidence for each outcome RCTs – high, Observational – Low Grading recommendations for practice - Problem priority - Acceptability - Benefits/harms - Preferences/values - Quality of evidence - Resource use - Feasibility - Equity Downgrading Upgrading - Risk of bias - Large effect - Inconsistency - Dose-response - Imprecision - All plausible - Indirectness residual - Publication bias confounding Quality of evidence is defined as the extent of our confidence that the estimates of the effect are correct Comparative effectiveness Meta-analysis of RCTs; narrative summary 1 2 3 4 5 6 DECIDE

115. Please email with questions/comments: CONSORT.study@spi.ox.ac.uk CONSORT.study@spi.ox.ac.uk Visit our website: http://tinyurl.com/CONSORT-study http://tinyurl.com/CONSORT-study Thank You!