Evidence Based Practice 3 Summarising papers and assessing their merits

Session Plan 2 tasks: Develop a structured summary for each paper Tea Break Assess the methodological merits of each paper

Why do a structured summary? Ensure you record the important information Give you an easily comparable summary of the aim, method, measures, findings and conclusions of each paper. To facilitate this it is easiest to use a template such as the one included in your pack.

Template 1 Full Reference Details of the paper: All authors Full title of the paper Publication Date Journal name Volume number and page numbers (OR doi) So that when you lose your copy you can find it again easily

Template 2 Summarise the aim of the study Summarise the study method Into one sentence if possible Summarise the study method Qualitative, case study series Quantitative, semi-structured interview Mixed methods, list Pilot study?

Template 3 Describe the sample selection procedures: Random /purposive Specific characteristics Gender Age limits Specific diagnosis or problem Number sought and reason for this How many participants? How was this number decided?

Template 4 Summarise what participants were asked to do This may be different if you have more than one group in the study What measures were made? How were these measures undertaken? Were any researchers involved in measurements?

Template 5 If the study involved randomisation how was this done? Who was aware of the grouping? If researchers were involved in measurements, were they blinded to the intervention? Were participants blinded to the intervention they received?

Template 6 How was data analysed? Summarise the main results Were the appropriate procedures used? In a qualitative study what procedures were used to ensure rigour in the data analysis? What happened if participants withdrew? Summarise the main results Concentrate on results that are important for answering your question

Template 7 What conclusions have the authors drawn? Are these reasonable given the study methods and procedures? Are these consistent with the results? Leave a space for inserting a ‘quality’ score This may be in sections for a qualitative paper or it may be numerical for quantitative papers.

Fill in one template for each of the papers you have located and read.

Assessing the merit of each paper We are not going to assess the quality of the writing or the writing style (this is usually heavily influenced by editorial staff of the journal). We are interested in noting the parts of each paper that contribute to improving the scientific merit of the study and therefore the weighting we would give to the study.

Selecting a framework to use Comparisons between papers are clearest when all papers have been assessed on the same points Value of the assessment is increased when all of the important methodological points are included in the assessment Different types of studies have different methods for reducing bias so the framework needs to be appropriate for the type of study

Possible frameworks PEDro GRADE PRISMA Downs and Black COSMIN AGREE QADAS Daly et al

PEDro 1. Eligibility criteria were specified 2. Random allocation 3. Concealed allocation 4. Baseline similarity between groups 5. Subject blinding 6. Therapist blinding 7. Assessor blinding 8. Follow-up > 85% 9. Intention-to-treat analysis 10. Between group statistical comparisons 11. Point measures and measures of variability reported. Item scoring: 1 = present, 0 = absent. Criteria 1 is not included in the total score

Identify the framework What framework is most likely to be applicable to the two papers you have summarised? Get a copy Read what you are asked to score to confirm that it is applicable

Using the framework Assess each of your 2 papers using the selected framework Record the outcome for each assessment point for each paper What does this indicate about each paper? How could each paper have been improved? Are there any points you think should have been included in the assessment that are missing?

Using the framework When you use any assessment framework consider: For each assessment item does the paper include the important information? In a randomised controlled trial are you given the method of randomisation? Is it a true randomisation? Are groups matched? Does the matching cover the important attributes?

Next time Pulling the results together Making tables and graphs Working out how to reconcile conflicting results Reporting your findings

Preparation Read any other papers you have located Make a structured summary for each paper Score each papers on the framework you have used Bring the structured summaries of all of your papers (including the ones you have done today)