1. Reviewer Training xxx zone

2. Welcome and introductions

3. Aims of Reviewer Training To promote understanding of the purpose and implementation of the National Cancer Peer Review Programme To provide peer reviewers with the skills and information to enable them to fully contribute and participate as equal team members in the National Cancer Peer Review Programme

4. Learning Outcomes Hopefully, by the end of the day you will: Understand the principles of the National Cancer Peer Review Programme and your responsibilities as a reviewer Understand your role and its relationship with other team members and the Zonal Team Understand what you need to do at each stage of the peer review visit Have practiced the skills you will need when reviewing Be familiar with and recognise the importance of the CQuINS electronic communications system

5. Session 1 The National Cancer Peer Review Programme

6. What is Cancer Peer Review? A process undertaken to assess the quality of cancer services against IOG for NHS patients in England, enabling quality improvement An integral part of the cancer reform strategy(2007) Supported by a set of detailed measures It is not a statutory function, but is mandatory i.e. commissioned by the SHAs through the DH

7. Purpose of Cancer Peer Review To ensure services are as safe as possibleTo improve the quality and effectiveness of careTo improve the patient and carer experienceTo undertake independent, fair reviews of services To provide development and learning for all involved To encourage the dissemination of good practice

8. Organisation of the Programme National Clinical Outcomes Group North Zonal Advisory Group Central Zonal Advisory Group London Zonal Advisory Group South Zonal Advisory Group

9. Quality Manager Quality Director Who’s Who? National Team Acting National Coordinator Ruth Bridgeman – Acting Deputy National Coordinator Julia Hill XXX Zone Team Clinical Leads xxx Admin Support xxx Assistant Quality Manager xxx Quality Mangers xxx Quality Director xxx

10. The Peer Review Programme Peer Review Visits Targeted Externally Verified Self- Assessments Sampled Validated Self-Assessments (annual) All Teams

11. The Measures Evidence based using NICE evidence linked to “Improving Outcomes Guidance” and agreed best practice based on national consensus Development of measures for each topic is undertaken by an expert group Consultation on new measures Measures coded as per year, topic and measure number e.g. 08 2J 102

12. Handbook for the National Cancer Peer Review Programme Contains details of the process from start to finish including: 1.The Peer Review Programme 2.Annual Self Assessment 3.Internal Validation 4.External Verification 5.National Schedule for Peer Review Visits & Conducting the Review 6.Outcomes of the Peer Review Process 7.Identification of Concerns 8.CQuINS

13. Session 2 The Cancer Peer Review Visit

14. What is a Peer Review Visit? Purpose: Provide an opportunity to meet with members of a service to determine compliance with the quality measures Identify any broader issues relating to the delivery of a quality and safe service Provide a further external check on internal quality assurance processes

15. What does a Peer Review Visit entail for Reviewers? PreparationVisiting a locality/networkWorking as a team to review evidenceIdentifying questions for clarification.Meeting with the teamDrawing conclusionsReport writingChecking draft report

16. Who are Reviewers? Multidisciplinary teams of Service Users, Clinicians, Managers, Commissioners “Peers are people who have been trained and working in the same discipline as the people they are reviewing” Reviewers will not normally review their own network

17. Peer Review Teams May Include: User/Carer MDT Lead Clinician Clinical Nurse Specialist Radiologist PathologistOncologist Medical Physicist Therapy Radiographer Oncology Pharmacist Chemotherapy Nurse Palliative Care Consultant Trust Lead Clinician, Nurse or Manager Network Lead Clinician, Nurse or Manager PCT Cancer Lead Cancer Commissioner Dietician

18. Selection Criteria for a Peer Review Visit New TeamsMilestones not met for implementation of an IOG as agreed with CAT Immediate Risks identified at previous peer review visits that have not yet been resolved Requests from organisations i.e. SHAs, Local and Specialist commissioners, PCTs, Networks, Acute Trusts % compliance with measures within lowest performance groupingConcerns regarding rigor of Internal ValidationStratified random sample based on % compliance (if available capacity)

19. Notification by 31 st December to teams to be peer reviewed during May - March Deadline for submission of evidence for all teams to be visited Pre-visit meeting for NEW TEAMS with the Zonal Team or Zonal Team Pre- assessment circulated Visits MAY-MARCH Each Network is allocated one month. Can take from 1 to 4 weeks to complete a Network – normally 1 day per Locality Report published 8 weeks after last review day December - 4 Weeks - 6 WEEKS The Peer Review Visit Plan Preparation for review + 8 WEEKS

20. Information for review team Practical arrangements Emergency contact details Membership of visiting team Access to key evidence documents Compliance against the measures Timetable Previous reports Interpretation guidance Cancer measures Expenses claim form Contextual information

21. Typical Timetable TimeAM ReviewsPM Reviews 08.30Arrive & Introductions 09.00Preparation 10.30Meeting with the Service - up to 3 concurrent sessions 11.30Arrive & Introductions and then preparation 12.30Lunch and report writingLunch 1.00Conclusions and report writingPreparation cont. 13.30Meeting with the Service - up to 3 concurrent sessions 3.00Depart 3.30Conclusions and report writing 5.00Zonal Feedback to Trust Lead Team and visiting team depart

22. Session 3 Review of Evidence

23. MDT Key Questions Can You : Demonstrate that you have a properly constituted and functioning MDT? Demonstrate that you have effective systems for providing coordinated care to individual patients? Demonstrate that your team has adequate information to help it improve service delivery? Demonstrate how you are continuously improving your service (including both clinical effectiveness and the patient experience)?

24. MDT Evidence Documents Operational Policy Annual Report Work Programme Team function Patient Pathway Polices and procedures Clinical guidelines and treatment protocols Achievements & challenges Use of data to assess service provision MDT Workload & Activity Data National Audits Local Audits Patient Feedback Trial Recruitment Work Programme Update Actions to address any issues Plans for service improvement & development

25. Relationship of Measures to Key Documents Information required to comply with the measures should be contained in the key documents (Evidence Guides were developed to support teams in the preparation of evidence) When compiling the report you will need to refer to the actual measures, not abbreviated versions or what you think the measure is! Look at the compliance section of the measure Decisions against the measures are made as a team If unsure then check with the Zonal Team

26. Compliance with the Measures Yes / No / N/A – at the time of the visit –All aspects of the measure have to be met Comments –No and … –No but … –Yes and … –Yes but … Content of policies, procedures etc. –Existence usually required by measure –Content determines yes/no only so far as is stipulated in the measure

27. Demonstrating Agreement Where agreement to guidelines and policies is required this should be stated clearly on the cover sheet of the relevant evidence document Evidence Guides will indicate the groups and individuals that need to be documented as agreeing the key evidence documents See Examples within Evidence Guides

28. Pilot for Breast and Lung Service 2010/2011 Details of Clinical Lines of Enquiry: Identification of clinical indicators Data Clinical lines of enquiry (See the Delivery Specification Guide on CQuINS and evidence guides).

29. Identification of Clinical Indicators Discussion with SSCRG Leads, NCIN and Medical Directors Outcome of the work has been to develop clinical indicators for Breast and Lung It is the intention to feedback and review the Clinical Indicators at the NCIN Site Specific Clinical Reference Groups on an annual basis

30. Data Where National data is available, this will be provided to both the review teams and MDTs A proforma will be provided to the Trusts for the collation of local data. It should be returned to the Zonal Teams and the National Cancer Peer Review Senior Information Analyst by the end of the self-assessment period The proforma for collation of data for breast cancer is available in the Delivery Specification Guide on CQuINS

31. Clinical Lines of Enquiry A briefing sheet on the relevance of the clinical indicators Structure and ensure consistency of the discussions about the data on a peer review visit A commentary on the clinical lines of enquiry should be included in the PR and IV reports.

32. Clinical Lines of Enquiry in Reports Question 3 for an NSSG: ‘Can you demonstrate that you have effective processes in place for evaluating services across the network and identifying priorities for improvement?’ Question 4 for MDTs: ‘Can you demonstrate how you are continuously improving your service?’

33. Reviewing Evidence Quality Measures Evidence Documents Key Questions

34. Reviewer Responsibilities

35. Working on behalf of the health community within the zone Preparation before the visit – information pack Reviewing the evidence and ascertaining compliance Identifying issues relating to the Measures and to the quality of service for patients Allocating questions for each reviewer Keeping to timetable Confidentiality...sensitivity Obtaining consensus re compliance/report content Writing the report Responsibilities include:

36. Approach Open, fair, consistent and comparable Encourage development and learning All members of the team have a valuable contribution to make –“No question is a silly question.” Try to understand and follow the patient’s journey

37. The Evidence – Preparation on the Day Welcome and introductions Review of evidence documents to –Establish a ‘feel’ for the service –Initial consensus on compliance –Review case notes if applicable – NHS professionals only –Identify and formulate questions Agree structure and process for the meeting

38. CQuINS

39. Using CQuINS V4 Available via the web site at: www.cquins.nhs.ukwww.cquins.nhs.uk Secure web based database supporting each stage of the cancer peer review process Records assessments, compliance with the measures and reports Provides information for national analysis and reporting

40. CQuINS In preparation for your visits you will need a CQuINS password in order to gain access to the following evidence: The teams self- assessments The teams externally verified reports User reviewers will be sent copies of the key documents

41. CQuINS Homepage Navigate to CQuINS homepage www.cquins.nhs.uk www.cquins.nhs.uk Enter your registration details

42. Viewing Key Evidence Documents (1) View or download each evidence document. Any internal validation reports will be available to view or download in this section % compliance at self assessment stage and internal validation stage (if applicable) will be detailed in this section. OP – Operational Policy AR – Annual Report WP – Work Programme APP – Additional appendices

43. Viewing Key Evidence Documents (2) If you select view the document will be displayed directly onto the screen if it is in a PDF format (recommended format for all documents uploaded to CQuINS). If it is a word or excel document a new page will be opened so that you can view it.

44. Group Work 1 – A Fictional Lung MDT- Preparing to Meet the Team You are reviewing the evidence for a Lung MDT and need to make an initial assessment. Using the spreadsheet, review the evidence for those measures highlighted Consider whether you have sufficient information for initial compliance and identify any questions you want to ask the team Complete recording proforma

45. Key Points from Group Work What conclusions did you reach after looking at the evidence? What questions do you have that you need to ask the team?

46. Session 4 Meeting with the Team

47. Question Types OpenEncourage people to talk What, where, when, why, how? ClosedChecking factsHow many? do you? HypotheticalTest contingenciesWhat if? ProbingGreater depthIn what way? Tell me more.. LeadingPacifying Use with care You do, don’t you? MultipleAvoidWho sends it, what do you do and how?

48. Funneling Questions Open - to explore Probing - to clarify Hypothetical - to test contingencies Closed - to confirm

49. Listening Listen before deciding response –Their story not yours! –Be active summarise regularly ask confirmatory questions –Encourage contributions from everyone –What if they won’t stop talking?

50. Body Language Look for reaction and adjust style to suit Watch for defensive behaviour sudden arm-crossing agitation face touching avoidance of eye-contact Use empathy

51. Which Team Members Should You Expect to Meet? Lead Clinician and CNS with other core members e.g. Surgeon, Oncologist, Radiologist, Pathologist, Palliative Care MDT Review Chair of Locality Group, with Lead Cancer Team, Commissioner, PCT and User Reps Locality Chair of Network Group Small group of other key group members inc. User Reps Network Group Review

52. Possible Challenges The need to deal with ‘politically’ or ‘organisationally’ sensitive issues The need to deal with immediate risksThe need to deal with a hostile organisation The need to manage appropriate engagement from the review team

53. Discussion Topics Recent improvements Tracking the patient pathway, referral, clinics, diagnostics, MDT treatment options, follow up, support (survivorship), discharge / death, links to palliative care General team issues, audit, service improvement, governance, clinical trial entry, data collection Relationship with NSSG / referring teams Current clinical service development issues

54. Meeting the Team Start on time and keep to time Make sure everyone is introduced Put at ease – and get comfortable Develop rapport Explain purpose and structure Note-taking Open questions to start

55. Closing the meeting Ideally, summarise the main issues Give the opportunity for further comment: –“Is there anything we’ve missed?” –“Is there anything else you’d like to tell us about?” –“Which aspect of the service are you most proud of?” Thanks If serious / sensitive issues – talk to Team Leader or Zonal Team ASAP

56. Summary Be prepared, know what you need to ask, why, how and by whom Clear introduction Open question to start Listen actively Take good notes Summarise Give chance for further comments Thanks

57. Group Work 2 – Part 1 A Fictional Lung MDT Divide into groups Formulate your questions arising from the previous group work Consider how they link to the 4 key MDT questions Think about whether there are any areas of concern

58. Group Work 2 – Part 2 Read the brief for your character Reviewers need to decide who is asking what, why and how MDT members need to decide on who will respond to any issues 10 mins. – Question planning 15 mins. – Role preparation 25 mins. – Meeting with the team (20 mins. + 5 mins. debrief) Return for feedback Then in groups we will cover completing report forms

59. Feedback from review meeting

60. Session 5 Completing Report Forms

61. Drawing Conclusions (1) Work together as team to produce an accurate, objective report – one scribe –All team members have a responsibility to contribute –Consensus view –Use the 4 key questions to support report writing Check each measure: –Yes / No / N/A –If no: Give reason why –If very good: Say so! Agree issues: –Always include MDT membership, attendance, cover –Quantify issues if possible –Write sentences …. or as near as possible Progress from last visit

62. Drawing Conclusions (2) Evaluate and describe the situation – avoid solution/recommendation Legible and evidenced General conclusions Clarify/explain the point being raised. –Waiting times are good/too long – The waiting times are currently three weeks for …. –Patient information is good/poor – There is no provision of any local patient information…

63. Sample Compliance Sheet

64. Categorising Review Findings An issue that is likely to result in harm and requires immediate action Immediate Risk An issue that could compromise the quality or outcome of patient care Serious Concern An issue that affects the delivery or quality of the service Concern Relates to the service and can be either innovative or common practice undertaken very well Good Practice

65. Exercise in your groups Identify which category each of the issues identified would fall into Group Work 3

66. Feedback on Concerns

67. Group Work 4 (30 mins.) Quickly check through the measures Any changes as a result of meeting with the team Any queries From the information you have gathered work together to categorise issues and document on the report proforma

68. Feedback from Group Work

69. PLEASE….. Do not take the final report away with you!

70. What Happens Next? Feedback Draft report circulated to Review Team and Trust / Network for comment on factual accuracy Final report published on CQuINS within 8 weeks. Reports will be publically available

71. Session 6 Evaluation and Close

72. Aims of Reviewer Training To promote understanding of the purpose and implementation of the National Cancer Peer Review Programme To provide peer reviewers with the skills and information to enable them to fully contribute and participate as equal team members in the National Cancer Peer Review Programme

73. Learning Outcomes Understand the principles of the National Cancer Peer Review Programme and your responsibilities as a reviewer Understand your role and its relationship with other team members and the Zonal Team Understand what you need to do at each stage of the peer review visit Have practiced the skills you will need when reviewing Be familiar with and recognise the importance of the CQuINS electronic communications system

74. Resources Available www.cquins.nhs.uk www.cquins.nhs.uk NCPR HandbookEvidence guides for each topic areaCQuINS help sectionSlides and packs from todayZonal Teams are able to offer support

75. Any Questions?