1. Training for Organisations participating in Peer Review of Paediatric Diabetes

2. Welcome and introductions

3. Aims of Today’s Training
To promote understanding of the purpose and implementation of the peer review of Paediatric Diabetes.
To enable organisations to be prepared for the forthcoming peer review of Paediatric Diabetes and ensure that all good practice is shared.
To enable organisations to cascade details of the review to others within their organisation.

4. Learning Outcomes Hopefully, by the end of the day you will:
Understand the principles of Peer Review and your responsibilities
Understand your role and its relationship with the Peer Review Co-ordinating Team
Understand what you need to do at each stage of the peer review visit
Have practiced the skills you will need when reviewing*
Be familiar with and recognise the importance of the CQuINS electronic communications system

5. Session 1
The Paediatric Diabetes
Peer Review Programme

6. The New Health Environment

7. Benefits of Peer Review
Provision of disease specific information across the country together with information about individual teams which has been externally validated
Provision of benchmark data
Provision of a catalyst for change and service improvement
Identification and resolution of immediate risks to patients and or staff
Engagement of a substantial number of front line clinicians in reviews

8. Benefits of Peer Review
Clinical support for a peer review
Enabled rapid sharing of learning between clinicians, as well as a better understanding of :
Commissioning Services for Children and Young People with Diabetes
Delivering the Diabetes National Service Framework
Provide support for future business cases

9. Aim of Peer Review Ensuring Services are as safe as possible
Improving the quality and effectiveness of care
Improving the patient and carer experience
Undertaking independent, fair reviews of services
Providing development and learning for all involved
Encouraging the dissemination of good practice

10. Our Key Principles Clinically Led
Focus on Coordination within and across organisations
Consistency in delivery of Programme
Developmental
Integration with other review systems
Peer on Peer
User/Carer Involvement

11. Setting the Future Direction
Peer Review Programme
NICE Quality Standards
Regional /National Programme Review
(Potentially)
CQC
Essential Standards
NHS Outcomes Framework
Patient Expectations

12. Externally Verified Self-Assessments Validated Self-Assessments
The Process
Peer Review
Visits
Targeted
Externally Verified Self-Assessments
Sampled
Validated Self-Assessments
(annual)
All Teams

13. Session 2 – Completing a Self Assessment

14. The New Self Assessment Process
Quality Measures
Evidence Documents
Key Questions
This part of the process hasn’t changed a great deal following consultation and piloting.
The New Self-assessment process includes:
-Key Questions for each topic area
-Evidence Documents that will provide the evidence against the quality measures
-A revised set of quality measures – with a number of measures that a no longer relevant being removed from the manual and other measures being merged together
NOTE TO SPEAKER – It is important that we explain how the key questions and key evidence documents link to the quality measures

15. What are “Key Questions”?
Clarify purpose of self-assessment
High-level summary of measures
Self-Assessment should address the 4 key questions relevant to that team
Form part of the Validation of Self Assessments
Lets first explain what we mean by the “Key Questions”.
Important to clarify that we are not asking for seperate evidence answering these – they should be seen as a tool to help teams think about their self-assessment and help teams “assess” themselves – “Where are our strengths and weaknesses against these key questions”.
When we talk later about how self-assessments are validated – they key questions will form part of that vlaidation process.

16. MDT Key Questions Can You:
Demonstrate that you have a properly constituted and functioning MDT?
Demonstrate that you have effective systems for providing coordinated care to individual patients?
Demonstrate that your team has adequate information to help it improve service delivery?
Demonstrate how you are continuously improving your service (including both clinical effectiveness and the patient experience)?
IMPORTANT HERE TO EXPLAIN THE THINKING BEHIND EACH QUESTION. SEE GUIDANCE BELOW. THIS IS CONTAINED WITHIN DELEGATE PACKS AND EVIDENCE GUIDES.
Can you demonstrate that you have a properly constituted and functioning MDT?
This can be demonstrated through compliance to those measures that relate to MDT Leadership, MDT Structure (membership) and MDT Meeting Arrangements (including attendance). In addition, measures within the operational policies section regarding ensuring all new patients are reviewed by the MDT, % time MDT Core members devote to this cancer type, training requirements of MDT members and responsibilities of nurse MDT Members also help demonstrate this. MDT Workload data and surgical workload data is also important here.
Can you demonstrate that you have effective systems for providing coordinated care to individual patients?
This can be demonstrated through compliance to those measures that relate to the existence of a coordinated and patient centred pathway of care. For example, measures relating to communication with patients, key worker and principal clinician policies, communication with GPs, gaining feedback from patients, recording of treatment planning decisions, and agreement of Network Clinical Guidelines.
Demonstration of coordinated referral pathways between specialist and local teams is also an important part of this.
In addition, teams may demonstrate within their evidence other aspects of service delivery not covered by the existing measures that fit in here (for example, the provision of streamlined diagnostic pathways, enhanced recovery programmes or other patient support initiatives).
Can you demonstrate that your team has adequate information to help it improve service delivery?
The term information is used in its broadest sense to cover data, audit, feedback from patients and feedback from service improvement initiatives.
Compliance to measures relating to data-collection (collection of agreed minimum data-sets for example), participation in agreed Network/National Audits, service improvement initiatives and gaining feedback from patients help demonstrate this question.
In addition, teams may demonstrate within their evidence other initiatives, over and above the existing measures, that give further assurance against this question. For example, audit activity, other initiatives to learn from the patients experience and innovative use of data. Access to accurate information about the teams workload is also important here.
Can you demonstrate how you are continuously improving your service (including clinical effectiveness and the patient experience)?
This follows on from the previous question. It relates to the implementation of improvements to the delivery of services on an ongoing basis. Demonstration of the outcomes from audit activity, implementation of actual measured improvements to services delivery and implementation of improvements to the patients experience of those services will give assurance that this question is met.
It is important to demonstrate the outcome or measurement of the improvement (whether it is related to the patients experience, clinical outcome, waiting times, or other quality indicators) within your evidence.

17. MDT- Evidence Documents
Operational Policy
Annual Report
Work Programme
Describing how the team functions and how care is delivered across the patient pathway
Outlining policies/processes that govern safe / high quality care
Agreement to and demonstration of the clinical guidelines and treatment protocols for team.
Summary assessment of achievements & challenges
Demonstration that the team is using available information (including data) to assess its own service
-MDT Workload & Activity Data
(activity by modality, surgical workload by surgeon, numbers discussed at MDT, MDT attendance)
-National Audits
-Local Audits
-Patient Feedback
-Trial Recruitment
-Work Programme Update
How the team is planning to address weaknesses and further develop its service.
Outline of the teams plans for service improvement & development over the coming year
-Audit Programme
-Patient feedback
-Actions from Previous reviews
Now lets talk about the evidence that’s required from teams to support a self-assessment. The difference with the new approach is that we want self-assessments to be “lighter” – rather than having to submit separate documentation against each individual measure, we will be asking for “evidence documents” – usually three documents – which should then demonstrate compliance against all of the quality measures. These are not documents that are written to satisfy the requirements of Peer Review – they are documents that any mature, functioning Cancer MDT should already have in place.
For an MDT we will expect to see an operational policy – outlining how the team is to function and the policies and processes that govern safe, quality care.
The Annual Report presents a report back over the previous year – showing MDT Attendance records, workload data, audits that have been undertaken, feedback from patients and also a brief narrative about your achievements and challenges over the previous year. Where data is available from National Audits on “Headline Indicators” Or Proxy Measures of Outcome we would like to see this reported in the Annual Report.
The Work Programme demonstrates the plans the team have in place to improve, to address weaknesses and further develop their service.

18. Demonstrating Agreement
Where agreement to guidelines and policies is required this should be stated clearly on the cover sheet of the relevant evidence document.
Evidence Guides will indicate the groups and individuals that need to be documented as agreeing the key evidence documents.
We are no longer requiring signatures to demonstrate “Agreement” to policies and guidelines. However, we do expect that each evidence document will have a cover sheet which states which groups and individuals have agreed the documents (or subsections of the document where appropriate).

19. Using our Evidence Guides
Guidance to help you structure your evidence documents
Guidance for Compliance
Additional Guidance
Always refer to the full measure in making assessments against measures
Evidence Guides will be published on CQuINS. There are seperate Evidence Guides for Each Topic Area.
[Refer to the Lung Evidence Guides – there should be a set in the delegate pack]
So if you are a Lung MDT you will have a guide for your Operational Policy, Annual Report and Work programme. They will help demonstrate which measures can be evidenced within which document. They include guidance for what should be included to comply with a measure but also additional guidance of what else should be in the document.
IMPORTANT TO CLARIFY – WHEN MAKING ASSESSMENTS AGAINST MEASURES YOU SHOULD ALWAYS REFER TO THE ACTUAL MEASURE NOT THE EVIDENCE GUIDE.

20. Self-Assessment – Key Tips:
Use the evidence guides
Get the evidence agreed in line with the measures
Ensure all Agreements are documented on evidence cover sheets
Be honest
Don’t let yourself down with poor evidence
Data Requirements for Annual Reports – establish a process
Sell yourself
Use Annual report – focus on Outcomes

21. Group Work – A Fictional Lung MDT
You are the Cancer Management team for the Trust and are required to validate the Lung MDT Self-Assessment.
You will need to review the teams evidence & for the measures highlighted in BLUE on the spreadsheet, identify whether you agree with the self-assessment or not.
Introduce Group Work

22. Group Work – A Fictional Lung MDT
Consider whether you have sufficient information or whether you would need to see any supporting documents or ask questions of the team?
Then look at the validation proforma based on your review of the evidence – focus on one of the 4 key questions....and also think about whether there are any areas of concern?
40 Minutes
NOTE TO FACILITATOR – ALLOCATE A SEPERATE KEY QUESTION FOR EACH TABLE TO VALIDATE.

23. Key Points from Group Work
Did you change compliance of any measures?
What questions would you like to ask this MDT?
How did you complete the “key questions”?
What are the Key Concerns
2C-102 – YES
2C-106 – NO – Cover is not listed for all team members
2C-107 – No – There is 0% attendance from Histopathologist and low attendance from Oncologists – THIS IS A SERIOUS CONCERN
2C-129 – YES
2c-130 – YES

24. Session 3 The Paediatric Diabetes Peer Review Visit
Throughout the day the sessions will cover the relevant theory and then some group work so that by the end of the day reviewers will know exactly what is expected when undertaking a peer review visit.

25. Who are Reviewers? Multidisciplinary teams of
Service Users, Clinicians, Managers, Commissioners
“Peers are people who have been trained and working in the same discipline as the people they are reviewing”
Reviewers will not be from the own Trust
Worth mentioning here, but also reiterated later.
Make sure that when we book you let us know if you think there are any possible conflicts of interests in order reassure both yourself and the team you are reviewing e.g.
You may have had a placement at the organisation and you don’t feel it is an appropriate review for you.
We have had incidences where family members are part of the team being reviewed or reviewers have had family members who have had personal dealings with the team.
We would ask that you exercise sensibility and remain objective.

26. Which Team Members Should You Expect to Meet?
Lead Clinician and CNS
with other core members e.g. Surgeon, Oncologist, Radiologist, Pathologist, Palliative Care
MDT Review
Chair of Locality Group,
with Lead Cancer Team, Commissioner, PCT and User Reps
Locality
Chair of Network Group
Small group of other key group members inc. User Reps
Network Group Review

27. The Visit Day Peer Review team Preparation 1.5 Hours
Peer Review Meeting with Team being reviewed
Peer Review Team Report Writing
1 Hour

28. Typical Timetable Time Review 08.30 Arrive & Introductions 09.00
Preparation
10.30
Meeting with the Service Team
11.30
12.30
Lunch and report writing
1.00
Conclusions and report writing
13.30
3.00
Depart
3.30
5.00
Coordinating Team Feedback to Trust Lead
This slide is to show reviewer what a typical day may look like and repeated later on to help focus delegates- Zonal team can amend slide to show their own timetable format if applicable
Site specific reviewer will concentrate on their relevant topic. Depending on the timetable core team members will be allocated to a specific review.
In some circumstances core team members may be required to stay all day.
Although the review module take 4.5 hrs, report writing can take longer than one hour - need to brief reviewers.

29. Categorising Review Findings
An issue that is likely to result in harm and requires immediate action
Immediate Risk
An issue that could compromise the quality or outcome of patient care
Serious Concern
An issue that affects the delivery or quality of the service
Concern
Relates to the service and can be either innovative or common practice undertaken very well
Good Practice
Talk through the catergorisation of issues and refer reviewers to the handbook and the risk matrix in their pack Good practice / best practice – recognising the work of the team and any areas that could be shared.
Missing core team members and cover are always a concern.

30. Handbook for the Paediatric Diabetes Peer Review Programme
Contains details of the process from start to finish
Including:
The Peer Review Programme
Self Assessment
Outcomes of the Peer Review Process
Identification of Concerns
CQuINS
The revised Handbook is available on CQuINS and covers all the elements of the Cancer Peer Review Programme.

31. CQuINS

32. Using CQuINS V4 Available via the web site at: www.cquins.nhs.uk
Secure web based database supporting each stage of the cancer peer review process
Records assessments, compliance with the measures and reports
Provides information for national analysis and reporting
Lead into the Cquins session -
This is how the evidence organised and managed to make it easily accessible for organisations and reviewers, so they are now live documents. Therefore the risk of looking at different version has been reduced. The advice to teams is that they should upload their documents in PDF format the documents allow easier viewing , shorter download and easier search functions.
Important to remind users that they will be allocated a temporary password , just before the visit, so that they can view evidence .(Some users may already have access if they are a member of a network group, this permission remains with the respective organisation).
Please advise delegates we will have a Laptop set up for anyone who wished to view the CD Rom system / live link over lunchtime

33. CQuINS
In preparation for your visits you will need a CQuINS password in order to gain access to the following evidence:
The teams self- assessments
The teams externally verified reports
User reviewers will be sent copies of the key documents
All NHS Reviewers will can register for a password on Cquins and then access their own organisations evidence and any unpublished information relating to the team they are reviewing. Once they have registered the organisational contact will receive an automated and will authorise access.
(Zonal teams will only be registering User Reviewers)
Include screen shots in the pack as will not be able to see detail in presentation handouts

34. CQuINS Homepage Navigate to CQuINS homepage www.cquins.nhs.uk
Enter your registration details
Also show reviewers where to register for a password .

35. This is an example of what a Lung MDTs “Home Page” will look like.
It will provide a quick summary of % compliance by assessment type. (blue table). This will also show the dates this team were last assessed and also the deadlines for when they next need to be assessed.
There will also be some summary “benchamrking” (graph) showing how this teams assessments compare to all other Lung MDTs in the country.
The Evidence Documents uploaded as part of this teams self-assessment are available for viewing or downloading from the home page (green section) as will be any reports written about this team (IV, EV or PR Reports).
The Team Members for this team will be listed on the home page as well (orange section).
On the left of the screen are navigation tools – so the list of Local MDTs are all the other MDTs within the same Trust as this team – so they can quickly look at how other MDTs within their Trust are doing – or they can click on “similar MDTs” to have a list of all other Lung MDTs within the country and have a look at their home pages and assessments.
Quick Find will allow you to search by typing in names of Trusts, locations, Networks etc etc,
The tabs along the top of the screen (grey tabs) allow you to access the key functions of CQuINS.
SIMILAR Home Pages are being developed for other “levels of user” each Trust or Locality Home page for Trust Cancer Management Teams, Network Home pages for Network Lead Teams and Zonal Home pages for Zonal Coordinating Teams.
There are three steps to completing a self-assessment, 1. Upload Evidence 2. Assess Compliance 3. Link Measures to the Evidence
SO CLICK ON “UPLOAD EVIDENCE” TAB

36. This allows you to upload all evidence at once
This allows you to upload all evidence at once. Simply click browse and locate the evidence on your PC.

37. Find the relevant document.

38. CQuINS will code each document by document type – so which ever document you upload under Opertational Policy will be coded as OP, Annual Repot as AR and Work Programme as WP – you can also give each doc your own title. CICK UPLOAD

39. The system will work better for you if your documents are uploaded in a PDF Format. This will allow users to quickly view the documents on the system without having to download them. CLICK ON VIEW

40. The document is viewable without having to be downloaded.
PDF Docs will also mean that when you have mapped your evidence to individual measures – when you click on a measure the specific part of you evidence will appear (this function is being developed as we speak so cant demontrate today)
PDF Software is free and can be quickly downloaded from the web – speak to your IT team – we will also give advice on this on CQUINS.
OK AFTER UPLOADING YOUR EVIDENCE YOU WILL THEN NEED TO ASSESS COMPLIANCE AGAINST THE MEASURES – SO CLICK ON THE ASSESSMENTS TAB

41. You will need to ensure that the drop down box says self-assessment (not validation) and.

42. To self assess simply click on the drop down list for each measure and click Y, N or N/A.

43. If you wich you may also want to enter a comment against particular measures.

44. If you want to see the full measure and compliance simply hover the mouse over the measure and the full measure will appear.

45. Once you have complete your assessment click on save assessemnt
Once you have complete your assessment click on save assessemnt. If you want to you can do half and then come back to it later. A self assessment can be changed at anytime up until the point its been validated or up until the deadline prior to a peer review visit.

46. Next step is to link each measure to your evidence
Next step is to link each measure to your evidence. To do this click on the attach documents tab.

48. For each measure you will then need to tick which document includes the evidence for compliance against that measure. If you want to – rather than ticking you can type in a page number. You can tick more than one document. If you uploaded other documents these would also appear as extra colons - headed “Other”
Once you have completed them all – click the “ATTACH” tab at the bottom of the screen.
PLEASE NOTE – THIS SCREEN WILL INCLUDE A REMINDER OF HOW YOU ASSESSED EACH MEASURE

49. If you then go back to View Assessments you get the full screen showing how you have assessed each measure and where the evidence is. This screen can them be downloaded into an excel spreadsheet.
THATS THE SELF-ASSESSMENT PROCESS – WHICH CAN BE COMPLETED IN ABOUT 10 MINUTES.

50. CQUINS V4 Front end Website Resources for Teams Measure Manuals
Published Reports
Public Information About NCPR.
NCPR News
Login Access to the Database
(which wont be publicly accessible)
The Version 4 Database will be accessed via a website (
This website will contain resources for teams (Evidence Guides and Measure Manuals for example) as well all published reports and public information about NCPR. Users can then log into the actual database from this website.

51. The Peer Review Visit Plan
December
- 4 Weeks
Preparation for
review
- 6 WEEKS
+ 8 WEEKS
Notification by 31st December to teams to be peer reviewed during May - March
Deadline for submission of evidence for all teams to be visited
Pre-visit meeting for NEW TEAMS with the Zonal Team or
Zonal Team Pre-assessment circulated
Visits
MAY-MARCH
Each Network is allocated one month. Can take from 1 to 4 weeks to complete a Network – normally 1 day per Locality
Report published 8 weeks after last review day
Explain briefly the process for notifying organisations. This is an annual process

52. What Happens Next? Feedback
Draft report circulated to Review Team and Trust / Network for comment on factual accuracy
Final report published on CQuINS within 8 weeks.
Reports will be publically available
Won’t be individual feedback at the end of each module
Will be some high level feedback by the zonal team to the Trust team at end of whole visit
Everyone will need to work together to ensure that reports are ready for publication. Lots more detail in handbook.
Once the report is tidied you will receive a copy to comment on the factual accuracy, before it is sent to the organisation . Please make sure that you respond even if you do not have any comments.

53. Resources Available www.cquins.nhs.uk
NCPR Handbook
Evidence guides for each topic area
CQuINS ‘resource’ section
Slides and packs from today
Zonal Teams are able to offer support

54. Any Questions? Thank you
Check flip chart with delegates to make sure that all expectations from the first session have been covered.