1. So how good are your results? (An introduction to quantitative QC)
Graham Jones
Chemical Pathology
St Vincent’s Hospital, Sydney

2. Overview Standard QC Quantitative QC Bias
How good are we?
How good do we need to be?
Bias
A short course in what we will need to know

3. Quality System Staff Instruments Sample management QC
Choosing
Training
Instruments
Optimising
Maintaining
Sample management QC
Planning
Performing
Responding
Quality Assurance
Performance
Interpretation
Action
Result management

4. Quality System Staff Instruments Sample management QC
Choosing
Training
Instruments
Optimising
Maintaining
Sample management QC
Planning
Performing
Responding
Quality Assurance
Performance
Interpretation
Action
Result management

5. Quality Terminology QA - Quality Assurance
Planned, systematic actions providing confidence that a quality output will be produced
Laboratory Procedures
QC - Quality Control (QC)
Procedures use to assess validity of results in real time, controls release of results
QC material run with patient samples
EQA - External Quality Assessment (PT)
Procedures operated by an external agency which allow retrospective review of performance
RCPA-AACB

6. Running a QC Program Selection of material (matrix)
Selection of levels (decision points)
Setting of targets and ranges
Decision on frequency (batch vs RA)
Decision on number of QC samples (n)
Decision on rules and interpretation
Response to out-of-range values
Quality planning

7. QC Quiz Your new trainee scientist asks you the following question:
“In our lab, how far can the results of an assay vary from the actual concentration in the sample?”
What is the answer:
+/- 1SD; 2 SD; 3 SD; 4 SD; 5 SD ?

8. Assay Characteristics
Stable assays:
Performance defined by mean and SD
QC never fails
Results “always” within +/- 2SD

9. Stable Assay Mean = 20, SD = 1 95% of QC results between 18 and 22.
Interpretation: A result of 20 has a 95% confidence interval of 18 to 22.

10. Unstable Assays Mean drifts over time (fluctuating bias)
QC process used to detect drifts
Variation in results due to scatter plus drift

11. Unstable Assay Mean = 20, SD = 1, Plus fluctuating mean.
Interpretation: Result of 20 has 95% confidence limit of
18 – 22, PLUS bias at time of assay.

12. Unstable assays
How bad can this be?
How can we measure this?

13. Original Westgard Multi-rules

14. Original Westgard Multi-rules

15. Original Westgard Multi-rules
What does “In Control” mean?

17. Power Function Chart N=2 13s/22s/R4s Probability of Rules Firing
Shift in Mean (multiples of SD)

18. Power Function Charts

19. Power Function Chart N=2 13s/22s/R4s 90% error detection at 3.2 x SD
Probability of Rules Firing
Shift in Mean (multiples of SD)
90% error detection at 3.2 x SD

20. Shifts and Results (unstable assay)
Imprecision: up to 2 SD.
Undetected shifts in mean: 3 SD
Total spread: up to 5 SD
With the assay still “in control”!
+3 SD
+2 SD
+5 SD

21. QC Quiz The result may differ from the correct result by addition of:
random variation of the assay (up to 2 x SD)
the undetected bias at the time of the assay (up to 3 x SD).
Example
CV cholesterol assay: 2.0%
At 6 mmol/L, 5 x SD = 0.6 mmol/L
Accumulation of errors all in the same direction is rare, but can happen.

22. Understanding our assays
For any assay, with the QC protocol in place, we should be able to say how much analytical error may occur.
“These rules have the power to cause a STOP 90% of the occasions when there is a shift in the assay of 2.8 x LSD and cause a PAUSE 90% of the occasions when there is a shift in the assay of 2.6 x LSD.”
- SydPath Quality Control SOP

23. Westgard - Quantifying QC
With the rules I have in place, what shifts in assay performance can I detect. or
How can I be sure that I can detect important changes
Capability
Setting QC protocols
What are “important changes”

24. Capability Capable assays are easily able to “do their job”
Capable assays (almost) never produce results outside important limits.
Poorly capable assays will produce results outside the set limits.
Capable assay
Incapable assay

25. Capability
Good assays (capable) have an analytical performance (SD) which is much less than the clinically important change.
This can be quantified as the Capability Index: Cp=ALP/SD
(ALP=Allowable limit of Performance)
>6 great; OK; <4 poor

26. Required shift detection
Capability
Required shift detection
Cp = 6, good
4 SD Shift
2 SD spread
Cp = 4, OK
2 SD Shift
Cp = 3, poor
1 SD Shift
Important Change

27. Capability
Capability is the concept we use to discuss quality of assay performance.
Relates assay precision to required precision.

28. Reverse Engineering QC
If we have limits to our assay performance we want QC protocols which allow us to detect assay problems before “wrong” results may be issued.
Choose QC protocols which allow appropriate error detection.
Use Power Function Charts….

29. Power Function Graph (n=2)
12s
12.5s MR
13.5s

30. Setting QC Protocols Capable assays – simple rules
Poorly capable assays – Need:
More rules
More QC
Incapable assays – will not achieve target performance
Have to accept less chance of finding shifts
Or choose better assay

31. Quality Specifications Hierarchy
How good do we need to be:
1. Proven analyte-specific data on clinical decision making
2. General-clinical decision making
Based on biological variation
Based on medical opinions
3. Professional recommendations
4. Regulations or EQA targets
5. Published state-of-the-art data

32. Within-subject biological variation
Variability in patient results due to changes in the patient.
Archives
Reference Material (near bottom on right)
Biological Variation Database
eg Sodium: 0.7%; ALT 24.3%

33. Analytical v Biological CV
CVa/CVb
Relative total CV

34. Percent increase in CV
CVa / CVb

35. Precision Targets
Within-person biological variation provides limits to benefits of improved assay precision
Many assays clearly capable (optimal)
Some assays not able to reach targets

36. Bias Most discussion thus far has related to precision
Attention to bias will be the next major issue

37. Summary (1) Some aspects of QC can be quantified
Power function charts are a tool
This process allows us to use appropriate QC
can either
Know how good we are or
Aim to reach certain targets

38. Summary (2) Targets can come from various sources
Within-person biological variation provides a useful reference point.
Bias will need to be addressed
Control of bias will provide many advantages