1. Using Quality Research strategies to improve a QI project Associate Professor Helen McBurney Department of Physiotherapy Monash University & Latrobe Regional Hospital Associate Professor School of Primary Health Care Monash University

2. Purpose of this workshop To help you identify strategies to improve the quality of your quality improvement project. The strategies are based on methods used to improve the quality of a research project

3. Identify your problem What is a current issue in your facility/practice? Is there an area where you have not reviewed your practices or procedures for a significant time? Is there a journal article that you have reviewed that uses an evidence based practice framework and suggests that your practices may be no longer the ‘best’? Is there a service delivery issue needing to be addressed?

4. Identify your problem Case Study 1 We need to have 50 more patients receive a joint replacement in the next 3 months than in the last 3 months. There will be no additional beds or staff time allocated for these procedures.

5. Questions arising from the problem Case Study 1 How many joint replacement patients were there in the past 3 months? What was the average length of stay? What was their discharge destination? What is best practice? What are reasonable actions or strategies we can use to address the problem?

6. Identify your problem Case study 2 Our HITH program is clearly above the State average for Length of Stay and for readmissions in DRG 64a & b

7. Questions arising from the problem Case Study 2 What is the State average (and SD or other statistics) for length of stay? What is the State average for readmissions? What is the best practice evidence available in the literature? What strategies can we adopt to address this problem?

8. Turn the problem into an answerable question PICO: The question should identify: the patient group or problem (P) the intervention (I) any comparison of interest (C) the outcome (s) of interest (O) The Comparison is optional all others should be included

9. Turn the problem into an answerable question Case 1 Does the use of the Risk Assessment and Prediction Tool (RAPT) facilitate safe and early discharge home from the acute ward after a hip or knee joint replacement?

10. Turn the problem into an answerable question Case 2 Where does the HITH service we offer for patients in DRG 64a and b differ from that of best practice or of a service with below State average length of stay and readmissions?

11. Identify standards for comparison What measures are to be used? These should be identified before you start These should be appropriate and realistic There should be a good reason for the selection of the comparison standard this could be a national benchmark or a comparison to your own outcomes

12. Identify standards for comparison Case 1 Measures: 1.Number of procedures: compared to last 3 months or the same 3 months last year 2.Length of Stay: comparison with Australia wide average available for the last year from the Australian Orthopaedic Association Joint Replacement Registry 3.Discharge destination: increase in numbers going directly home compared to last 3 months 4.Safety: number of readmissions within 28 days for related issue (fall, wound infection etc)

13. Identify standards for comparison Case 2 What describes best practice in the literature? What is the State average for length of stay for this DRG? What is the average for readmissions? Can we identify another facility like us in size and patient numbers but doing better in LOS and readmissions to compare our policies and procedures?

14. What resources are needed? Personnel Time Equipment Data analysis (many health facilities do not have easy access to statistics programs for staff) Are these available? Costs? – don’t forget to add on costs

15. What resources are needed? Case 1 File auditor with time to review >200 records. RAPT Data analysis program All found internally – cost borne by the organisation

16. What resources are needed? Case 2 Personnel with skills to: –Perform literature review –Audit files –Analyse state-wide data –Liaise with other facilities Computer and Library access: –Access on-line data bases and gather journal papers –Spread sheet for audit –Data analysis –Report writing

17. What resources are needed? Case 2 Telephone / e-mail/ visit to distant health facility by staff Staff time and travel costs Not able to be met internally – successfully applied for a grant from the Department of Health

18. Do you need approval? Management? Good idea – sell your project as a QI winner Ethics? Depends on what you are doing and your actual method – talk to the Secretary of your ethics cttee Read NH&MRC guidelines: National Statement on Ethical Conduct in Human Research When does quality assurance in health care require independent ethical review? Read Health Commissioners guidelines regarding information privacy

19. Approval? Case 1 Application for ethics waiver granted on the basis that: The processes were routinely used at other hospitals Data collection would be undertaken by a staff member who would usually have access to the information No individual was able to be identified in the reporting

20. Approval? Case 2 Application for ethics waiver granted on the basis that: The audit was retrospective (no treatment would be changed) Data collection would be undertaken by a staff member who would usually have access to the information State data was de-identifed Any other facility involved was given the opportunity to refuse to participate

21. Gathering data 1: Method Data collection methods: File audit Questionnaire Survey Interview Observation Measure Method needs to be: Decided first Appropriate Valid Reliable Easily administered Easily understood Best solution might involve multiple methods

22. Gathering data 2: What data? What information is needed? In what format? Example: To get age, would you ask for Date of birth Age at a specified date Age in a category (eg 15-20, 20-25 etc) What are the advantages and disadvantages of each method?

23. Gathering data: Resources Identify the resources needed for your data collection in more detail For questionnaires or surveys: how many copies, how many pages, how will they be distributed & returned, who will computerise the responses, short response or multiple choice or a combination, how will analysis be conducted For interviews: For audits: Measurements:

24. Gathering data 3: Source Identify the important sources of data People: –patients, –carers, –therapists, –managers, –students, –therapy assistants Decide sample size –power analysis? Information: –journal papers –medical records –local / state / commonwealth government reports or data bases (ABS) –other databases held by NGOs such as AOA, NHF

25. Managing Data Concerns: privacy and/or confidentiality accuracy retention Get data into computer format compatible with analysis program as soon as possible De-identify as soon as possible

26. Data Management: Case 1 File audit data (past 3 months) entered to excel spread sheet as raw data and as RAPT scores Data directly imported into SPSS Same spread sheet used for data entry for all patients over the next 3 months

27. Data Management: Case 2 Literature review used standard procedures to record databases searched & search outcomes, inclusions and exclusions, quality review procedures File Audit set up as an excel database with direct entry of data by auditing staff member VAED data Received as an excel file with >12,500 cases and imported into SPSS Comparison with other hospital Notes taken for all contacts and information

28. Data Analysis Quantitative Frequencies Level of data: Nominal (young, old) Ordinal (0-4, 5-9, 10-14) Continuous (actual age) Number of groups for comparison Number of time frames for comparison Select appropriate form of statistical analysis Qualitative Transcribe responses accurately Analyse for common ideas or themes Independent analysis by more than 1 person Discuss and agree findings Disagreement resolved by?

29. Data Analysis Case 1 Audit all cases from previous 3 months Collect RAPT at pre-admission for all cases admitted in next 3 months Outcomes – number of hip or knee joint replacements, LOS, discharge destination, readmission in <30 days

30. Data Analysis Case 2 Quantitative Data from file audit Match audit data to VAED –Compare LOS and readmission rates Analyse issues identified in audit –Coding queries Identify other hospital for comparison –LOS –readmissions Qualitative Description of policy and procedures at Hospital A and B Analysis of similarities and differences Analysis of differences in context Possibilities for changes Recommendations to the organisation

31. What do the results mean? Quantitative Statistical significance –Has the intervention caused a change in the mean score of the group for an outcome? Clinical Importance –Is the mean change that has occurred important enough in the ‘real world’ to suggest that the intervention should be adopted as normal practice? Qualitative Describe what your participants think Describe their feelings Describe their perceptions Collect their ideas for service improvements

32. Results Case 1 Jan –March, 71 patients 63% female 41 H: 19 L,22 R 30 K:15 L,15 R Mean age70.25 DestinationLOS acute Home 45%6.38 Rehab 55%7.38 Readmit4.2% (3) April – June, 120 patients 56% female 59 H: 13 L,46 R 61 K: 24 L,37 R Mean age 68.29 DestinationLOS acute Home 57%6.11 Rehab 43%6.66 Readmit4.2% (5)

33. Results: Case 2 Literature review reveals few studies that have tested the same antibiotics and few that have randomised home versus hospital care – ‘best practice’ is difficult to identify – patients clearly have a preference for home based care. Audit of files shows 276 patients with DRG 64a or 64b in one year as hospital n=124, home n=115 or mixed n=37 treatment.

34. Results: Case 2 State wide data shows 12,510 patients with DRG 64a or 64b in one year as hospital n=9,764, home n=1,604 or mixed n=1142 treatment. State wide mean HITH LOS 6.23 days Our mean HITH LOS 7.56 days Comparison hospital HITH LOS 5.22 days

35. Results: Case 2 Between hospital program differences: Policy: Procedures: staffing, travel, patient selection antibiotic selection, delivery doses, delivery routes, review procedures

36. How do we change our practice? Based on the evidence, do we need to change? –policy –procedures –train personnel – get new equipment What can we do to ensure any change is sustainable? –make it easy for everyone

37. Change in practice – Case 1 Use of RAPT score at preadmission is now routine and guides advice regarding time in hospital and likely discharge destination More patients go directly home No increase in readmissions Higher level of activity in acute ward Reduced waiting list

38. Change in practice - Case 2 Results are open for discussion between the research team, the HITH team and the Quality unit of the hospital Options include: – using the data to persuade management of the need for greater consistency in staffing for the HITH unit –reviewing coding by the HIMU

39. Where to from here? Next session in Ararat on 5 th September At this session we will consider how to report the results –To management –As an abstract for a conference –As a conference paper or poster –As a journal paper

40. Where to from here? Work on your project and bring your results along Questions: e-mail me helen.mcburney@monash.edu