2. Introduction to Critical Appraisal This section presents a framework for use in the critical appraisal of research papers and encourages an exploration of the many frameworks available. The framework included here is employed to illustrate the critical appraisal of a paper that includes both qualitative and quantitative methods, though concentrates on the analysis of the approaches taken to data analysis. Applying a framework of questions to interrogation a paper allows the reader to critically appraise the work, identifying its strengths and weaknesses and providing opportunities to make informed judgements about the study.

3. Introduction to Critical Appraisal This section presents a framework for use in the critical appraisal of research papers and encourages an exploration of the many frameworks available. The critical appraisal of published research as part of a literature review can inform the development of research questions, hypothesis and methodological approaches, or confirm that the body of knowledge that exists is sufficiently robust to suggest further research is not required or vice versa. A systematic review of the literature differs from a literature review, being based on a scientific design, which aims to reduce bias and increase reliability and provide a comprehensive picture of all of the available evidence.

4. Frameworks for Critically Appraising Research Articles The Introduction Is there a clear statement about the topic being investigated? Is there a clear rationale for the research? Is there a clear statement about the limitations of the research? The Literature Review Do the researchers use contemporary material about the topic being investigated? Do the researchers link their work to a wider body of knowledge through the references cited? Do the researchers link the topic to the questions about theory? Is there a clear link between the literature and the formulation of the research question(s)? Is the research question clearly stated?

5. Frameworks for Critically Appraising Research Articles The Methods Section Is the research design clearly described? Are the research methods appropriate for the topic being investigated? Are any advantages or disadvantages of the design acknowledged by the researchers? For researchs which involve respondents for sampling Is there a clear statement about who participated in the research? Is there a clear statement about how the participants were selected? Is the selection of participants appropriate to the design? Is there a clear statement about the number of people taking part in the research?

6. Frameworks for Critically Appraising Research Articles Data Collection and Analysis Is there a clear description about how the data was collected? Was the data collected by appropriate instruments/people? Is the approach to data analysis appropriate to the type of data collected? Quantitative Is there any explanation of sample size used? Is the level of significance of the tests (alpha) used indicated, or implied to be the customary 5%? If Pearson correlation coefficients are being calculated, is there any evidence of a check for a linear relationship? If t-tests or analysis of variance (ANOVA) are to be preformed, is there any evidence of check(s) to demonstrate that the data follows a normal distribution, or of assumptions made? Are reasons/assumptions concerning the level of measurement of the data given? (This affects the appropriateness of the descriptive statistics given and the tests used.) Is there a clear statement describing how valid and reliable the measures are? Are the type of statistical tests used appropriate for the sorts of data collected? Is the use of any statistical analysis package, such as SPSS discussed?

7. Frameworks for Critically Appraising Research Articles Qualitative Is there a clear reflexive statement about the researcher's role in the analysis? Is the approach taken to data analysis clear? Is the use of any electronic analysis package discussed? Is there a clear statement about how the researcher validated interpretations? Ethics Is there a clear statement about ethical committee approval? Is there a clear description about gaining consent, maintaining anonymity and or confidentiality?

8. Frameworks for Critically Appraising Research Articles The Results/ Findings Are the results related back to the literature review? Are the weaknesses in research design acknowledged? Quantitative Is the presentation of results clear and unambiguous? Are all the results presented? Do the tables and charts used give a clear picture of the sample data and results? Are the charts used appropriate? Are the tables easy to use? If percentages are recorded, are actual numbers also clearly shown? Are results of tests interpreted rightly? Qualitative Does the research present evidence of the data collected? Does the data presented as part of a theme support the analysis suggested? Is there a clear audit trail?

9. Frameworks for Critically Appraising Research Articles The Conclusions Are the implications for further research acknowledged? Are areas for further research identified? Are further recommendations made for practice that come from the results/discussion?

10. Worked Example Introduction The development of the CD-ROM is presented and its importance as a nursing tool within the nursing curriculum is given. The project aim clearly states that the effectiveness of the CD-ROM as a learning aid will be measured, along with the experiences of students and lecturers using the CD-ROM. The rationale for the research is set in the context of a tool to support learning of a compulsory skill, ie BLS, and also to enable development of IT skills. However, the need for this tool is not stated in the introductory paragraphs.

11. Worked Example Literature Review A brief review is included in the introduction. This concentrates on referencing previous pilot evaluation work and relates to the importance of developing BLS skills. It does not identify other literature relating to BLS skill development or e-learning. Some quite dated material is referred to in terms of IT skills. The literature leads to a specific research aim rather than a research question

12. Worked Example Methods There is a statement of the stages of data collection, but no formal discussion of research design or alternatives. Strengths and weaknesses are not presented. The testing of BLS knowledge and skill attainment in the way conducted would seem to be a suitable way to gain information about the impact of the resource on BLS knowledge and skill attainment. Interviewing is a data collection method that is often used by researchers to ascertain individual experiences of events and would therefore be appropriate.

13. Worked Example Methods (cont’) The sampling approach is clearly laid out, showing institutional and individual involvement. There was random selection of the research sites, although there is no explanation as to why only 2 of the potential 13 sites were chosen for testing. Some limitations relating to students' prior experience of BLS skills are noted. The researcher states that students self-selected themselves into the focus groups and the research groups, which may lead to potential bias, as identified in the study. It is not clear how lecturers were involved in the research. Given the focus of developing BLS skills in the nursing curriculum it would seem appropriate to use this sample group.

14. Worked Example Data collection and analysis The data was collected by a member of university staff, with listed related publications, showing they have research experience in the field. It is not clear what the status of the researcher is or whether they hold a researcher role. Quantitative Data analysis is clearly described, including measures taken to validate data entry. Data was assumed to be at ordinal level, but no justification was given for this. However, assuming this, the appropriate paired non- parametric test (Wilcoxon) was used. No mention is made of the level of significance used, but it is assumed that the researcher was working at the 5% level. It is not clear how the 2 variables expired air respiration (EAR) and external chest compressions (ECC) were arrived at, but since they are given as percentages, it seems appropriate to carry out Pearson's correlation test. The linear relationship also required for this calculation is indicated in Figure 3 and 4. There is evidence of a statistician's input to the research, although there is no reference to any computer package used.

15. Worked Example Qualitative The researcher doesn't provide reflexive comment about her role in the research, though the approach to data preparation and analysis is clear. The researcher followed a recognised published approach to data reduction, display and verification, which on a positive note included independent researcher and respondent verification, thus improving trustworthiness of the data. Ethics The researcher states ethical approval was gained from each institution and a protocol to maintain informed consent, confidentiality and anonymity was applied, though unfortunately no detail of this is given.

16. Worked Example Results/findings Quantitative The tables and charts are clearly presented, although a table of results to include maximum, minimum and median values may have made comparisons between groups easier. However, this information is presented implicitly in the box-plots. There was no evidence of significant correlation between the variables EAR and ECC, with both probabilities being > 0.05. Indeed, one of the P-values is quoted as 1.738, which is clearly incorrect.

17. Worked Example Results/findings Qualitative The discussion of the four emergent themes is supported by verbatim extracts from respondents across both sites, to include lecturers and students, though it is not clear which individual's views are being presented. This could mean that the views of only a limited number of the sample are being presented. The extracts clearly support the themes presented. A range of published literature supports the themes and builds on the initial brief literature review. The researcher does not explicitly relate the work to a framework for ensuring trustworthiness.

18. Worked Example Conclusions The final paragraphs identify the limitations of the research before stating the key findings and relates well back to the aims of the project. Key points for higher education providers to consider are identified. The research fails to identify further research issues/opportunities arising from the study.