1. Using Primary Care Data for Quality Improvement Dr John Derry Primary Care Medical Adviser Thames Valley SHA

2. Oxfordshire MAAG 2 Preview Examples of clinical audit data –Similar to QMAS data Understanding variation –The role of SPC Using SPC methods to interpret clinical audit data

3. Oxfordshire MAAG 3 CHD Audit Results age & sex standardised prevalence of CVD

4. Oxfordshire MAAG 4 CHD Audit Results blood pressure recording & control

5. Oxfordshire MAAG 5 So what now? See variation What is significant? Is it OK to be near average? When should we act? How should we act? Which data can we use?

6. Oxfordshire MAAG 6 Use of “SPC” “Statistical Process Control” Methods developed by Shewhart and Deming (1930’s - 1990’s) Cornerstone of quality improvement Two different kinds of variation can affect any process Distinguish by statistical methods

7. Oxfordshire MAAG 7 “Statistical Process Control” “First and foremost, a way of thinking with some tools attached” “About the continual improvement of processes and outcomes” “About getting the most from your processes” Quotes from Don Wheeler in “Understanding Variation” SPC Press, 2000

8. Oxfordshire MAAG 8 Two types of variation How long does it normally take you to get to work? Why does it vary? How do you use this understanding to plan your journey? –When to leave the house –Which route to take –When to make a change

9. Oxfordshire MAAG 9 Understanding variation “Routine” “common causes” many factors, some “unknowable” “noise in the system” affects process most of the time part of the process variation is predictable “Exceptional” “special causes” “assignable” causes usually few, not many can usually be identified not part of the process intermittently apparent unpredictable variation

10. Oxfordshire MAAG 10 What to do about variation “Routine” don’t react to individual results look at the average and process limits improve the whole process if these not acceptable or continuously improve quality! “Exceptional” investigate each point outside the limits look for the special cause and do something about it almost always something to find opportunities to learn

11. Oxfordshire MAAG 11 Two kinds of mistake Mistake 1 Act as if there is a special cause when there is only routine variation –Might make things worse –Wasted effort anyway Mistake 2 Fail to spot a special cause – assume there is just routine variation present –Missed opportunity reduce variation improve quality learn something

12. Oxfordshire MAAG 12 Graphical method developed by Shewhart to help distinguish these two kinds of variation –routine and exceptional –predictable and unpredictable –common and special cause “Process behaviour charts” (Don Wheeler) Control charts

13. Oxfordshire MAAG 13 An example control chart Average “exceptional” variation “routine” variation

14. Oxfordshire MAAG 14 Another type of control chart Control Chart of Clinical Audit Data 0 5 10 15 20 25 0.0020.0040.0060.0080.00100.00 SqRt number without criterion SqRt number with criterion Plus 3 Sigma Minus 3 Sigma Average Routine variation Exceptional variation

15. Oxfordshire MAAG 15 How to interpret the chart Control Chart of Clinical Audit Data 0 5 10 15 20 25 0.0020.0040.0060.0080.00100.00 SqRt number without criterion SqRt number with criterion Practices here cannot be distinguished from average Practices here are significantly different from average Increasing difference Practices here are significantly different from average Increasing difference

16. Oxfordshire MAAG 16 “Double square-root chart” Described recently by Mohammed et al. (Lancet 2001; 357: 463-67) Originally developed by Fisher, Tukey & Mosteller in 1940’s Enable analysis of variation in “cross- sectional” data Based on binomial probability distribution for calculating SD

17. Oxfordshire MAAG 17 Types of control chart For measurement (variable) data –Single observations Limits based on average moving range Average +/- (3/bias correction factor d2) x average MR –Subgroups of observations Limits based on std deviation of subgroup Correction factor depends on number in each subgroup

18. Oxfordshire MAAG 18 Types of control chart Count (attribute) data –“Yes/no”, “with/without” data P-chart Limits based on Binomial conditions Average p +/- 3 x sqrt (p(1-p)/n) –“event” data count (needlestick injuries) U or C chart (denominator varies or constant) Limits based on Poisson conditions

19. Oxfordshire MAAG 19 Some real examples Using clinical audit data

20. Oxfordshire MAAG 20 434 429 424 420 417 416 413 405 406 407 409 0.00 5.00 10.00 15.00 20.00 25.00 0.0020.0040.0060.0080.00100.00 SqRt number without CVD SqRt number with CVD Standardised CVD Prevalence Average = 4.6% +/- 3SD Range = 3.6-5.7%

21. Oxfordshire MAAG 21 CHD Audit Results age & sex standardised prevalence of CVD

22. Oxfordshire MAAG 22 BP recorded 434 430 422 417 416 411 412 0.00 5.00 10.00 15.00 20.00 0.002.004.006.008.0010.0012.00 SqRt number without BP record SqRt number with BP record Average = 76% +/- 3SD Range = 61-84%

23. Oxfordshire MAAG 23 CHD Audit Results blood pressure recording & control

24. Oxfordshire MAAG 24 Audit Results Cardiovascular Disease Prevalence

25. Oxfordshire MAAG 25 Audit Results CVD Patients with cholesterol record

26. Oxfordshire MAAG 26 Audit Results Cholesterol levels in CVD Patients

27. Oxfordshire MAAG 27 Audit Results Statin Rx for CVD Patients

28. Oxfordshire MAAG 28 Control charts for clinical audit To answer the question –“What do we do now we’ve got the results?” To identify where to target efforts To know when to act To know what kind of action to take

29. Oxfordshire MAAG 29 Issues to consider Using the right kind of chart for the data Time-series data is generally better Limitations of binomial charts –Binomial conditions (are they met?) Probability of single item possessing the attribute is constant Each item is independent of others

30. Oxfordshire MAAG 30 Recommended Reading “Improving Healthcare with Control Charts: Basic and Advanced SPC Methods and Case Studies” by Raymond G. Carey ISBN – 0-87389-562-2 American Society for Quality Quality Press, 2003 www.asq.orgwww.asq.orgAvailable at Amazon!