1. RFT 2003-02 Evaluation of clinical interventions in community pharmacy Final Report This project was funded by the Australian Government Department of Health and Ageing as part of the Third Community Pharmacy Agreement Research and Development Program, which is managed by The Pharmacy Guild of Australia

2. 2 The Research Team Gregory Peterson Peter Tenni Helen Kruup Omar Hasan Brita Pekarsky James Reeve Michael Roberts Roger Rumble Julie Stokes

3. 3 RFT 2003-02: Evaluation of clinical interventions in community pharmacy Clinical Intervention –Where a pharmacist identifies, or is presented with, an actual or potential drug related problem and he or she recommends an action to be taken to resolve or prevent the problem

4. 4 Outline of Todays Presentation Methods –Recruitment, training –Evaluation of value Results –Frequency, Types, Drugs involved –Economic Analysis Conclusions Where to from here? Methods Conclusions Results

5. 5 Overview of Methods Methods

6. 6 Pharmacy Recruitment and Enrolment 250 WiniFRED ® pharmacies invited to participate 75 enrolled for the project, only 52 possible due to software, hardware or location issues Arms –Remuneration –Intervention Prompt –Observation Methods

7. 7 Remuneration Randomisation Methods Crossover design

8. 8 PROMISe Observers Seven observers, each visiting 3 pharmacies 9 times in 3 weeks –21 pharmacies observed Assist with documentation –Identify opportunities for documentation –Aid classification/documentation process Time some events –Investigation of problems, phone calls to doctors, discussions with patients Methods

9. 9 Automatic Intervention Prompt Related to antiplatelet prophylaxis for vascular events in diabetic patients Activated when oral antidiabetic agents were dispensed 31 of 52 pharmacies randomised to receive prompt Methods

10. 10 WiniFRED Interface Training Three training evenings plus initiation visits to each site Methods

11. 11 Online Classification Training All pharmacists who indicated that they would participate were required to complete an on-line training package –20 scenarios to be classified –2 case-based clinical skills assessments Pharmacist demographics questionnaire completed at this point 20 scenarios re-classified after 3 weeks of use of the system Methods

12. 12 PROMISe Data Collection Pharmacy Demographics –Daily workload and staffing –Entrepreneurial orientation –Prescriptions dispensed Pharmacist Demographics –Clinical skills –Job satisfaction Clinical Intervention Parameters –Patient demographics –Drug involved and other drugs taken by patient –Type of problem –Action taken, Recommendation made –Acceptance of recommendation –Reactive or proactive –Time taken –Documenting pharmacists rating of clinical significance Methods

13. 13 Data Collection Initially planned for 3 weeks Extended to 4, then 8 weeks to obtain sufficient numbers of interventions Loss of interest from many pharmacies once observation phase was over and project team left Melbourne. Methods

14. 14 Assessment of Value: Probability and Severity Considered Considerations:How Addressed in Method 1. The nature and severity of the potential consequence(s) had the intervention not occurred 2. The probability that the consequence(s) will occur before the intervention 3. The probability that the consequence(s) will occur despite (after) the intervention 4. The degree to which the intervention can be attributed to the pharmacist 1. Consequences table Economic and non-economic parameters for each level of severity, validated by experts, multiple consequences (positive and negative) possible 2,3. Panel members considered probability and severity for each consequence selected before and after the intervention 4. Panel members provided a value for attribution Methods

15. 15 Methods Consequences Table

16. 16 Assessment of Value PROMISe Economic Simulation and Extrapolation Model Methods

17. 17 Assessment of Value 16 Clinical Assessors in 4 panels –2 physicians, 6 GPs, 8 pharmacists Secure internet access to intervention details Each panel assessed the same set of 51 common interventions and a panel specific set of 60 randomly selected interventions –51 common interventions and 240 randomly selected interventions were assessed Methods

18. 18 Clinical Panel Intervention Display Methods

19. 19 Clinical Panel Selection of Consequences Methods

20. 20 Economic Analysis: Derivation of Main Value Indicators Methods

21. 21 Sources of Information Results 5. PROMISe SQL Database (~13,000 interventions and ~430,000 prescriptions) 3. PROMISe Pharmacist Demographics (125) 6. Clinical Panel Assessments (16 members, ~290 interventions) 7. PROMISe Pharmacists Feedback (~80) 1. PROMISe Pharmacy Demographics (52) 2. Non- PROMISe WiniFRED Pharmacy Demographics (~40) 8. Non-PROMISe Pharmacists Opinions (~400 phone interviews) 4. Direct Observation Visits (63 visits)

22. 22 Methods Results

23. 23 PROMISe Pharmacy Demographics (n = 52) Entrepreneurial Orientation –~15% more innovative (self determined from responses to 2 questions) Location and Size –No different to Non-PROMISe and non-WiniFRED pharmacies Date of QCPP Accreditation –More Innovators (accredited before December 1999) 5/48;10.4% cf 2.5% in Victoria IT Facilities and resources –Used to determine attitudes to skills in IT area Daily Staffing levels –Used for workload analysis and simulations Results

24. 24 PROMISe Pharmacist Demographics (n= 125) Gender, Age, Registration year –Younger age group (80/125; 64% <40yo) Practice Profile –17/122; 14% accredited for medication reviews (cf ~5%) Scenario Classification Score (Before and after study) –Improved from 76% to 83% post trial Scored well for –Job Satisfaction (83%), –Professional Integrity (77%), –Change Readiness (73%) Clinical Skills –Good range of scores Results

25. 25 PROMISe Database: Non Clinical Interventions 11,493 Non-Clinical (Brand Substitution interventions) from 305,519 scripts (average rate of 3.7%) Under-utilised by pharmacists in study, still extrapolates to ~$15M pa Results

26. 26 Clinical Interventions Results 2396 interventions from 435,520 scripts 0.55 interventions per 100 scripts

27. 27 Clinical Interventions Results Observers Present Project Team Present Remuneration Decline in recording of interventions

28. 28 Clinical Interventions: Categories Results

29. 29 Clinical Interventions: Actions Results Investigation and discussion with patient common (71%) 18% contact with prescriber

30. 30 Clinical Interventions: Recommendations Results

31. 31 Clinical Interventions: Acceptance of Recommendations Results Dose, Drug or Education category interventions highly accepted

32. 32 Clinical Interventions: Proactive vs Reactive Drug selection, Dose problems more likely to be proactive Education and Toxicity less likely to be proactive (direct patient requests) Results

33. 33 Clinical Interventions: Clinical Significance (Pharmacist reported) More likely to be drug selection or toxicity problems and result in referral to GP Results

34. 34 Clinical Interventions: Drugs Involved- Numbers Results

35. 35 Clinical Interventions: Drugs Involved- Rates Skewed by intervention prompt Results

36. 36 Clinical Interventions for particular groups of drugs : Antidiabetic Agents Skewed by intervention prompt Results

37. 37 Clinical Interventions: Drugs Involved- Rates Results

38. 38 Results of Randomisation Results 52 Pharmacies Enrolled 22 Ever Observed30 Unobserved 12 Paid in Phase 1,3 18 Paid in Phase 2,3 11 Paid in Phase 1,3 11 Paid in Phase 2,3 9 Aspirin Prompt 5 No Aspirin Prompt 6* No Aspirin Prompt 2 No Aspirin Prompt 5 Aspirin Prompt 9 Aspirin Prompt 7 Aspirin Prompt 9 Aspirin Prompt *1 pharmacy converted to prompt after 1 week

39. 39 Clinical Interventions: Effect of Remuneration Results Effect of remuneration in first two weeks of study (univariate) Small impact when payment instituted (ameliorated reduction cf 20% reduction from Phase 1 to 2) 25% reduction 14%

40. 40 Clinical Interventions: Effect of Observation Observation significant (unadjusted) –1.02 vs 0.46 for all three phases (ever observed) –2.02 vs 0.8 for Phase 1 (observed days) Results

41. 41 Aspirin Intervention Prompt Effectiveness Total 194/7895 = 2.46 Aspirin prompt 193/4174 = 4.63 No prompt 1/ 3721 = 0.03 Ever observed 0.03 Never observed 0 Ever observed 157/2128 = 7.34 Never observed 37/2046 = 1.81 Aspirin interventions per 100 diabetic patients Only 7 in phase 3 (when aspirin prompt switched off) Results

42. 42 Aspirin Interventions: Observer Effect Phase 1: Observers PresentPhase 2: Observers Absent 12.6 1.84 2.3 1.3 Phase 3: Prompt off Results

43. 43 Clinical Interventions: Effect of Intervention Prompt Significant effect in first half of study on other interventions as well Results

44. 44 Aspirin Intervention Prompt No aspirin interventions without prompt Observation had a marked effect on increasing rate Increased rate of other interventions as well No residual effect of prompt –7 after prompt turned off ?Fatigue to prompt after 3-4 weeks Results

45. 45 Clinical Interventions: Multivariate Analysis of Effects Within Phases Results 2,384 Pharmacy Days where more than 20 prescriptions dispensed Mean Intervention Rate 0.74 (Standard Deviation 1.78) Phase 1 1.01 (1.79) Phase 2 0.88 (1.73) Phase 3 0.52 (1.76) Observed 2.02 (2.26) Observed 2.40 (2.87) Unobserved 0.78 (1.59) Unobserved 0.52 (1.76) Unobserved 0.80 (1.61) Paid 2.29 (2.43) Unpaid 1.68 (2.00) Paid 0.98 (1.94) Unpaid 0.65 (1.27) Paid 2.05 (2.29) Unpaid 2.67 (3.29) Paid 0.66 (1.37) Unpaid 0.91 (1.80) Paid 0.52 (1.76) Aspirin 2.45 (2.59) No Aspirin 1.56 (1.27) Aspirin 1.95 (2.17) No Aspirin 1.29 (1.69) Aspirin 1.31 (2.29) No Aspirin 0.48 (1.05) Aspirin 0.66 (1.32) No Aspirin 0.63 (1.18) Aspirin 2.90 (2.51) No Aspirin 1.20 (1.81) Aspirin 3.07 (3.58) No Aspirin 1.10 (0.70) Aspirin0.74 (1.40) No Aspirin 0.56 (1.33) Aspirin1.27 (2.12) No Aspirin 0.26 (0.52) Aspirin 0.81 (2.98) No Aspirin 0.42 (1.10) Phase, Prompt, Observation independently significant, payment not

46. 46 Clinical Interventions: Multivariate Analysis of Effects Within Phases Results No relationship effects when analysed by phase Phase One –Prompt significant (F=8.87; p = 0.003) –Observation significant (F=26.4; p <0.001) –Payment not significant (F=2.41; p = 0.121) Phase Two –Prompt significant (F=14.6; p < 0.001) –Observation significant (F=18.4; p <0.001) –Payment not significant (F=0.05; p = 0.813) Phase Three –Prompt significant ( F= 10.1; p = 0.001)

47. 47 Economic Analysis Overview Results Involves complex simulation of outcomes based on variable assumptions regarding: –The rate of interventions in observed vs non-observed days –The rates of interventions on busy and less busy days –The rates of interventions with and without the intervention prompt –The proportion of interventions performed that were actually documented (recording rate) Study design allows for estimate of opportunity for intervention

48. 48 Economic Analysis - Current Value Results All Australian pharmacies 232M scripts pa Average Value of Interventions in PROMISe data 0.22 days in hospital 1.23 consultations $290 in total costs 44 days of poor health Value of interventions in all Australian pharmacies 262,424 days in hospital 1.48M consultations $349M in total costs 53M days of poor health PROMISe Sample 52 Pharmacies for 8 weeks 2396 Interventions 435,000 prescriptions PROMISe intervention data 2373 Interventions 420,152 scripts PROMISe Assessed Sample 291 Interventions 1779 Assessments 16 Assessors PROMISe Assessed Sample 291 Interventions Clinical Assessment Process 1.6M interventions

49. 49 Economic Analysis: Assumptions in Current Value Simulation Results Recording rate on observed days 90% –Higher rate reduces final value Recording rate on Unobserved Days 50% –Higher rate reduced final value Attribution Rate 75% –Lower rate reduces final value Intervention Rate 0.69 per 100 scripts

50. 50 Economic Analysis: Varying Assumptions in Current Value Simulation Results

51. 51 Effect of Activity (Workload) Increased workload decreased the intervention rate by 75% for all interventions (even in the same pharmacies) 34/52; 65% Results Representation of pharmacies across each quintile 45/52; 87%46/52; 88%47/52; 90%42/52; 81%

52. 52 Effect of Activity and Intervention Prompt Uptake of educational prompt was relatively resistant to increased workload Results

53. 53 Economic Analysis: Assessing the Increased Opportunity for Intervention by Improving Staffing levels Results Based on intervention rate from second lowest quintile (45 of 52 pharmacies represented ; 87%) 28 hours of extra pharmacist time per 1000 prescriptions (~Cost $1300; ~ savings $910)

54. 54 Economic Analysis: Assessing the Increased Opportunity for Intervention by an Aspirin Prompt (or similar) Results Based on intervention rate achieved in Aspirin pop- up arm with observation –Observation mimics an education/ incentive program Additional $319M

55. 55 Economic Analysis: Assessing the Increased Opportunity for Intervention by Optimal Identification Results Additional $606M Based on 2.08 ints/100 scripts Achievable with suitable motivation…

56. 56 Motivators and Barriers for clinical interventions Key motivators were identified –Work environment –Clinical knowledge/continuing education –Professional satisfaction –Information continuity –Remuneration Barriers to documentation –Lack of time –Forgetfulness –Workflow restrictions –Software concerns Results

57. 57 Potential to increase interventions In the feedback on project, participants indicated that they would like to be able to carry out more interventions –Adequate staffing and staff mix –Continuing education –Identification of recordable incidents could be further optimised Results

58. 58 Conclusions Current value of clinical interventions is high Considerable scope for increasing intervention rate (and value) with educational techniques –Up to threefold Conclusions

59. 59 Conclusions

60. 60 Potential Roll-out Strategies for PROMISe Integrated whole solution approach –Repository model, –Feedback based on information received (education and quantified), –Pharmacist access to individual results and examples –Push information to pharmacies Incentive payment structure associated with targets for interventions Conclusions

61. 61 Recommendations Explore additional educational alerts Expand economic analysis to other datasets More detailed economic analysis to evaluate different types of interventions Identify factors associated with increased intervention rates –Workload Obtain more representative information –Larger sample for longer period Simplify and modify classification system for more widespread use Conclusions