1. Laboratory QC: Bridging the Gap Between Theory and Practice
11th Annual HuQAS Scientific Conference
Kenya Institute of Education
Nairobi, Kenya
September 29th, 2011

2. Post-analytical Results filing – correct result for correct vial
Result reported to correct number of decimals
Results filed after deadline
Results changed after reporting
RI reported with result
Comments reported

3. The Forces of Change Evidence based medicine - improve outcomes
Standardize medical management – Clinical Guidelines – tied to discrete test results (NGSP, NCEP-ATP, NKDEP)
Greater awareness of the costs associated with laboratory error and non-standardized testing
Electronic medical record – “the holy grail”
Accreditation Tied to Quality Management Systems
(ISO 15189, 17025, 17011)

4. Appreciating Today’s Reality
“trueness” is now becoming the analytical goal (JCTLM)
Peer group monitoring of performance rewards precision – “as long as I am consistently wrong I am ok”
How good do we need to be?
(patient centric clinically relevant performance goals)
How can we harmonize/standardize testing country wide?

5. Appreciating Today’s Reality
Assess laboratory performance using commutable test samples with target values assigned by reference methods
For success there is a need for laboratories to work together and to openly share their performance data on a network wide basis
Move towards the “ideal” as the performance of laboratory testing within the country improves

6. Are your methods good enough? What is good enough? How do you know?

7. “Performance is a measure of how we are doing” “Quality is a measure of performance against a defined standard or requirement” “Setting a performance standard is a pre-requisite for measurement of quality”

8. Clinical Laboratory Improvement Act (USA)
Laboratories must establish or verify Performance Standards for each test. They may get these either from the manufacturer (for unmodified methods) or establish them (for modified methods) (Section (b)(3)).
Laboratories are required to make these same performance standards available to clients upon request. ( (3)).

9. Quality Management of the Testing Process
A definition of what is expected (performance standards, SOP, process maps)
A system that measures variation from the expected (PT/EQA, IQC, internal auditing systems)
A strategy for effecting and sustaining desired change (non-conformance, root cause analysis, corrective action, preventive action)
A system for measuring and monitoring the impact of change (PT, internal auditing, outcome analysis, continual improvement) over time

11. Setting Quality Specifications
Global strategy for setting quality specifications in laboratory medicine (IUPAC, WHO, IFCC – Stockholm Scan J Clin Lab Invest 1999;57: ) 1. Quality specifications based upon an assessment of the effect of analytical performance on specific clinical decisions (NCEP-ATP) 2. Quality specifications based on biological variation (NKDEP- eGFR)

12. Setting Quality Specifications
3.Quality specifications based on medical opinions 4. Quality specifications based on guidelines from national/international expert groups 5. Quality specifications based on guidelines from expert individuals or institutional groups 6.Quality specifications as required by regulatory authorities 7. Quality specifications as laid down by PT/EQAS providers 8.Quality specifications as derived from published data on the state of the art

13. Setting Quality Specifications
1. Quality specifications based upon an assessment of the effect of analytical performance on specific clinical decisions Example: NCEP-ATP recommended performance specifications for the measurement of lipids

14. Setting Quality Specifications
NCEP Quality Specifications - Measurement of Total Cholesterol
Bias = 3% or less
(relative to A-K reference method)
Precision - CV = ≤ 3 %
TE (%) = bias x CV = 9% (rounded)

17. NCEP Performance goal (TE)
An International study examining the “trueness” and precision of lipid measurements in 27 countries
Analyte
NCEP Performance goal (TE)
% Failure
Total cholesterol
9 %
22%
Triglycerides
15% 7%
HDL
13%
48%
LDL
12%
33%

18. Comparison of Quality Specifications
NCEP
Criteria
CLIA
Cholesterol
RV +/- 9%
PG +/- 10%
HDL
RV +/- 13%
PG +/- 30%
LDL
RV +/- 12%
Not Regulated
Triglycerides
RV +/- 15%
PG +/- 25%

19. Setting Quality Specifications
2. General quality specifications based on biological variation
Example
National Kidney Disease Education Program – routine reporting of eGFR by the MDRD formula (NKDEP- eGFR)

20. Creatinine – Performance Goals
Recommended Total Error performance goals (TEa) for the measurement of creatinine as calculated on the basis of biological variation:
Minimum 11.4 %
Desirable 7.6%
Optimal 3.8%
Clin Chem 52:5-18 (2006)

21. Creatinine (CLIA Performance Criteria)
Peer group mean +/- 15% or 0.26 mg/dL
(23.4 umol/L) whichever is greater
Under CLIA – given a peer mean of 1.13 mg/dL
(100 umol/L) labs will get a “pass” if they are within +/- 23.4%
Clinically relevant TE performance goal for measurement of creatinine:
Minimum 11.4 %; desirable 7.6%; optimal 3.8%

22. Performance data from an accuracy based EQA program for the measurement of creatinine
Performance Assessment Criteria
RV +/- 7.6%
N= 176
IDMS Reference (umoles/L)
(mg/dL)
113.5
1.28
97.2
1.10
65.4
0.74
% of labs failing
7.4%
14.8%
38.2%

23. Setting Quality Specifications
3. General quality specifications based on medical opinions "If the true value for this test is "x", how close to this value must the reported result be in order for you to make a meaningful decision?" You will be lucky if you can get two clinicians to agree!

24. Setting Quality Specifications
4. Guidelines from national/international expert groups
“the clinical decision levels for albuminuria have been set
and applied without regard to the analytical method that
is being used in the determinations”
The performance specifications for the methods used in
these guidelines are typically not reported.

25. Setting Quality Specifications
5. Quality specifications laid down by regulation (CLIA) “CLIA quality specifications have more to do with what is achievable as opposed to what is actually needed to meet the clinical need “

26. Setting Quality Specifications
(6) Quality specifications as defined by EQA/PT providers

27. Assessment Criteria from Six PT/EQA Providers Quantitative – UrineB C D E F
Albumin
Peer Group Mean
+/- 30% or
3 SD
+/-10 mg/L
25%
30%
+/- 3 SD
+/- 2.5 SD
Creatinine
+/-30 mg/dL
15%
+/- 2 SD
+/- 0.3 mg/dL
17%
Albumin : Creatinine
Ratio
Not
Assessed

28. Fall-back Position Tonks' Rule
Tonks'’ Rule was one of the first published approaches for the setting of performance standards.
He suggested that the variation of a method should be less than 1/4 the span of the reference interval, divided by the mean of the reference interval.

30. “Biological Variation: from Principles to Practice” Callum G
“Biological Variation: from Principles to Practice” Callum G. Fraser AACC Press (2001) The most recent and extensive data on biological variation may be found at:

31. Setting Performance Criteria on the Basis of Biological Variation
“What is biological variation?”
Daily biological rhythms
Monthly cycles
Seasonal rhythms
Random biological variation

32. Biological Variation
“measure creatinine repeatedly in ten healthy male subjects over a period of 14 days”

33. Creatinine Reference Interval (64-120 umoles/L or 0.72-1.36 mg/dL)

34. Observations on Biological Variation
The range of values for anyone individual covers a small part of the reference interval
The mean values of all individuals lie within the reference interval and differ from each other
Within-subject variation is the quantitative estimate of homeostasis in man
Ample evidence to indicate that within-subject variation is constant
Little evidence that within-subject variation changes with age

35. Observations on Biological Variation
Homeostatic set points can change with age and pathology but the variation around these set points does not change
In general, estimates of CVi are constant irrespective of the number of subjects, the time scale of the study, the methodology and the country in which the study was done
Performance goals for precision and bias can be defined on the basis of within and between subject biological variation

36. Performance Criteria - Biological Variation
Three levels of performance can be defined: minimum, desirable and optimal
Performance Criteria for Precision
CVa < 0.25 Cvi (Optimal) (3% increase in Result Variability)
CVa < 0.50 CVi (Desirable) (12% increase in Result Variability)
CVa < 0.75 CVi (Minimum) (25% increase in Result Variability)
Desirable Performance Criteria for Bias
BA = <0.125 (CVi2 + CVg2)1/2 (2% increase in # people outside RI)
BA = <0.250 (CVi2 + CVg2)1/2 (16% “ “ “ “ RI)
BA = <0.375 (CVi2 + CVg2)1/2 (34% “ “ “ “ RI)

37. Performance Criteria - Biological Variation
The precision and bias goals can be combined to produce a desirable total error performance goal (TEa). TEa <0.250 (CVi2 + CVg2)1/ (0.50 CVi) In the case of sodium, the desirable TEa performance goal is 0.9%

39. “Six Sigma Quality Design & Control” Second Edition Desirable Precision and Requisite QC for Laboratory Measurement Processes James O. Westgard, PhD Westgard QC, Inc.

40. Six Sigma
“ A six sigma process is world class – with an expected error rate of 3.4 DPM (defects per million)” Requires that method bias plus 6s < total error allowable Total cholesterol (TEa = 9%) Bias from EQA/PT program = 1% CV = 1.5% Calculate sigma value: TEa – Bias /CV Sigma = 5.3 The performance of this method requires QC of only 2 controls per run with 3s or 2.5s control limits

41. Benchmarking with Six Sigma
“ A six sigma process is world class – with an expected error rate of 3.4 DPM (defects per million)” “Five sigma – is the initial goal for quality improvement - method requires QC of only 2 controls per run with 3s or 2.5s control limits” “Four sigma – method requires QC of 4 controls per run with multi-QC rules” (4000 DPM – 0.4% error rate; airline baggage handling – 4.15) “Three sigma – minimal acceptable quality - method requires maximum QC by laboratory – often a level that is not affordable for routine operations”

42. Failed PT – Now What?
Manager to staff: “To address this problem we must use root cause analysis. I’ll begin the discussion by saying that its not my fault”

44. Problem Solving – Root Cause Analysis
Define the problem
Gather data and evidence – pre-analytical, analytical, post-analytical
Ask "why" - identify the causes associated with each step in the process that contributed to the problem
Classify causal factors that relate to an event in the process, and root causes, that if applied can be agreed to have interrupted the process
Identify corrective action(s) that will with certainty prevent recurrence of the problem
Identify solutions with group consensus that will effectively prevent recurrence of the problem with reasonable certainty
Implement the recommended root cause correction
Confirm effectiveness by auditing the recommended solutions

45. Pre-analytical Sample received on time Sample stored properly
Sample condition
Sample matrix appropriate
Sample ID logged properly
Tests to be ordered
Method registration – check
Reagent dating, shelf-lie, storage
Reporting Units
Check maintenance logs, trouble shooting logs
Check non-conformities for the method involved

46. Analytical Method change recently Change in reagent lot
Recent maintenance on the analyzer
IQC running mean – has it shifted
Has CV of the method changed?
Is current CV meeting the set performance standard

47. Post-analytical Results filing – correct result for correct vial
Result reported to correct number of decimals
Results filed after deadline
Results changed after reporting
RI reported with result
Comments reported

49. David W Seccombe MD, PhD, FRCPC
Thank you
David W Seccombe MD, PhD, FRCPC
Department of Pathology and Laboratory Medicine,
University of British Columbia
Canadian External Quality Assessment Laboratory
(CEQAL)
DigitalPT International EQA Collaboration
(DigitalPT)