How to use you audit and indicator results for analysis and action (click on the ▼button on your computer to continue this presentation)

Did you know? Audit and indicator reports need to be reviewed and discussed by the service staff and management. The generation of the report is not the end- reports that are generated from indicator and audit results are one step of the action and analysis process.

The PDCA cycle and MOA Plan DoCheck/Study Act Feedback

Plan Check the calendar and the Quality Plan for audits and indicators due each month

Do Complete the audits. Collect and collate the Indicators

Check results using the reports Determine where action may be required

Act Use the action sheet to identify any issues arising from audits and indicator results. These issues may be identified from completion of the audits and collation of indicators. Use this as a tool for all staff completing audits to identify issues. It can also be used as a “brain-dump” during team meetings for any issues, not just those identified in the MOA program. You can copy these sheets and give them out to the person co-ordinating individual audits for them to assist you in identifying issues. Staff such as the Home Services Supervisors, Therapy Co-ordinators and Team Leaders should be co-ordinating completion and collection of audits.

Act Use the MOA Audit and Indicator Analysis Report to summarise issues and improvements with indicators, as well as audit results. This report goes to the Manager Area Operations on a monthly basis. This report can also be used by the service for its management team meetings, and as a tool to demonstrate to staff what happens with the information they assist us to collect. This report asks the service to comment on whether the indicator data has improved, is stable, or if issues have been identified. This demonstrates that the service is monitoring results on an ongoing basis, and not just acting on what has happened that month. Any issues identified requiring a quality activity to be commenced should be cross referenced to your QI Plan and the detail placed there, not in this report.

Act Check the Quality Plan for this month to see that: –All audits required have been completed –Indicators collected for this month have been reviewed –Actions have been implemented where required –Progress on actions from previous months have been reviewed –Suggested activities have been either completed in the month, or there is data/month scheduled for their completion –All relevant quality activities have been entered into the CI plan and progress noted.

Analysis of reports This section is for all managers, senior staff and other staff who may wish to learn more about how to get the most out of your MOA reports. You can apply these principles to any audits and indicators you collect. Don’t forget to read the MOA CQI guide for further information about report interpretation.

A note IRT commenced using the Moving ON Audits program in residential care in mid Community will be implementing the program in mid One in five services in Australia use the MOA program. Many have used the program for years, which means they have identified issues using these audits, and then use them each year to validate that their systems are still working. IRT is just starting on the journey, so it is natural that our results will not always compare favourably. When we find there is a significant difference-we have to ask the hard questions about whether we could do better. Don’t think of the program as something that just has to be done, think of it as a tool to help us do things better, and work towards our mission…

A premier aged care organisation Never –assumes everyone else is lying or cheating on their audits (and that is why they are achieving 100%) –ticks “yes” because it means less work and less questions to answer from their manager –thinks resloving issues identified is someone else’s job –keeps good ideas to themselves and does not share them with other services in the organisation –expects the administration staff to complete their incident forms for them. –blames other departments for issues identified in their audits –thinks that completing incidents forms and audits is a waste of time –uses programs like MOA as an excuse as to why their continuous improvement plan is not working

A premier aged care organisation Always –Takes time to explain the value of collecting indicators and completing audits to the staff who do the work, and involves them in reviewing the results –Validates audit results to make sure 100% really is 100% –Follows up on unusual incident data results or patterns –Takes the opportunity to use programs like MOA to improve people’s knowledge about each other’s roles –Celebrate its achievements and thanks its staff for doing such a great job

Let’s start Let’s start reviewing reports! Find an audit report, an indicator monthly report, and an indicator trending report, and review them with the guidelines on the next pages

Audit reports Your member number- member results are in red Your 1 st benchmark is TYPE- according to the type of service, eg care type, size and location. Results are in white. Your 2 nd benchmark is a comparison to all services in IRT with the same care type-results are in grey This is the GROUP benchmark.

Audit reports Your overall score The average for your service type The average for all members on MOA The average for NSW members- important to look at if the audit contains questions that can be interpreted differently across states The group score- in this case all low care in IRT

Audit reports Q Which questions require further attention/investigation?

Audit reports Question 7 -the service scored 100%,(a yes response) but the group benchmark average for IRT Low Care is 30%. What could be the reasons for this? Is there something the service is are doing that others in IRT are not? Have they misinterpreted the question? The other issue to note is that the type benchmark average is almost 87%. Removing IRT services from this benchmark, there are 38 other services across Australia where the majority have answered yes for this question. Should IRT look further into this issue-why is there average significantly lower? What does the audit explanation sheet say about this question? Question 8- the service scored 0%-which means they answered no for this question. The type average is almost 83%, but again the group benchmark average is significantly lower, at 20%. Again, is this something that IRT need to look into further.? What does the audit explanation sheet say? These are the type of issues that should be raised at area managers meetings.

Audit report Q What is the issue with the result for this question? The issue is that the average response for both other benchmark groups is higher than this score. The benchmark averages are not particularly high (which indicates others might struggle with this issue) but the score for this service is low. This question forms part B of the medication management audit, which means of all resident files audited (at least 5) none of the files had the evidence required. Any responses like this should be the focus of further discussion and investigation by the service. Ask others in IRT how they do it, look at the audit explanation sheets for further guidance.

Your audit report Have you identified any issues in your audit report?

Tips on interpreting reports Audits-100% result If you have achieved 100% as a result for an audit-congratulate yourself AND ensure that you can demonstrate that this result is valid by: Randomly checking completed audits on files by other staff; Randomly checking environmental areas they may have audited; Asking new staff to complete parts of the audit-and ask them to go and find the evidence-training and validation in one action! If you can’t validate the result, you need to state this in your moa analysis report, along with corrective actions. Otherwise, what will you say when something goes wrong, or someone outside the service finds out?

Indicator Reports Now find a monthly indicator report and review it along with the example provided.

Indicator reports The benchmark groups were described earlier in this presentation. The focus in these reports is indicator values (numbers) above the type and group median Remember indicators are just that-an indication that something might need further investigating, or you may just need to explain what is happening that is not typical in your service. As in this example, result of 2 existing wound infections (not healed) is not that more than having 0, but is it significant for your service. What may it indicate?

What is normal or stable? Have a look at your previous month’s data using your trending report. Look at the details of the incidents using the IRT clinical indicators database-you can generate a report by incident type, by month and also by individual resident.

Trending report Looking at the trending report for this indicator, there are no significant increases. The regular reporting of one indicates there may be a chronic/ongoing wound infection for one particular resident. If so, this is how the issue is discussed in the MOA analysis report.

MOA Analysis Report

Relationships and patterns Look for and comment on relationships between Falls with injury and new skin tears-identifiable cause-more often than not they may match up. UTI’s, RTIs and.. –Falls –Aggressive behaviour –Absconding (can be reported under an extra row in the report-it is not reported as an MOA indicator) Medication errors increase, and medication competencies being completed that month More new skin tears from transferring residents and new staff, and/or identified need for manual handling refresher training.

Every indicator does not require comment If the result for the month is stable and there are no new issues or significant issues, state this and move onto the next indicator. How do you know what is stable and normal? –Have a look at your trending reports –Review our data against the benchmark groups-if your data is consistently higher–ask yourselves why? What is different about your service and/or the residents at your service? Are a couple of residents’ multiple incidents influencing your overall results? –Look back at the definition of the indicator-are you applying it correctly, or over reporting?

Your indicator report What have you identified in your report that requires further action?

Action required-Area level Audit and indicator results should form part of the discussion at area meetings. It may be decided that there is an issue with a current policy or practice that requires review. This should be referred to the relevant committee that “owns” this policy. To find this out, check the Policy Register. Provide the committee with a summary of the issue and your area’s thoughts about how it could be improved. Ask for feedback from the Committee Chair so that you can then act on any recommendations relevant to you, and also update this in your MOA analysis sheet and CI plan as required. It may be decided that issues relating to Operations that may require further discussion may be referred to the Operations Leadership Team.

Action required-Portfolio level Your area managers meeting may identify that there is an issue that requires a response or clarification from another department. In this case, your Manager Area Operations will discuss this at the Operations Leadership meeting and if there is agreement that there is an issue, the relevant department or position will be provided with the relevant information to provide a response. Do not put a position or role down as being responsible to action an issue in your MOA analysis sheets or CI plan without discussing this with them first. It is simply not professional.

Problems with accessing or printing your package? Contact MOA during business hours on: | Phone:

Lets check what knowledge you have gained … Who should review audit and indicator results? How can I use the Plan, Do, Check and Act cycle to understand the MOA program at IRT? Who is responsible for following up on actions from indicators and audits? How do I interpret an audit report? How do I interpret an indicator report? What do I do with the results? What do premier aged care organisations do? How do I use the MOA Action sheet and Quality Plan? How do I use the MOA Analysis and Action Report? If there is a common issue among services identified in an audit, how does it get resolved?

Goodluck You can do it!