1. Utilization of Assay Performance Characteristics to Estimate Hemoglobin A 1c Result Reliability A. Woodworth, N. Korpi-Steiner, J.J. Miller, L.V. Rao, J. Yundt-Pacheco, L. Kuchipudi, C.A. Parvin, J.M. Rhea, and R. Molinaro August 2014 www.clinchem.org/content/60/8/1073.full © Copyright 2014 by the American Association for Clinical Chemistry

2. © Copyright 2009 by the American Association for Clinical Chemistry Introduction – Quality Control and Risk  Analytical Quality Control (QC) Programs are developed in all CLIA Certified Laboratories to rapidly identify and correct errors.  CMS Recently Introduced a new type of QC Plan – Individualized QC Plan (IQCP)  Laboratories can decide to use traditional QC approaches, or  Risk management strategies  The risk of reporting an unreliable patient result can be determined using analytical performance requirements and characteristics of an individual assay.

3. © Copyright 2009 by the American Association for Clinical Chemistry Introduction  A risk based IQCP for Hemoglobin (Hb) A 1c  Ideal to pilot a risk analysis because: −Harmonized and Standardized −Defined performance characteristics −Both lab and POC platforms −Large body of clinical information guides testing  This Hb A 1c risk assessment utilizes: −Assay performance characteristics in clinical settings −Three different acceptance limits ( ±5%, 6%, and 7%) −Standard QC practice ( 1-2S with 2 levels of QC) −Same number of patients between QC events (100)

4. © Copyright 2009 by the American Association for Clinical Chemistry Quantitative Tools Utilized for Risk Assessment  Assay imprecision (%CV) and accuracy (%Bias)  Allowable Total Error (TE a ) - Quality specifications for acceptable differences between the true and measured results for a given assay  Patient Weighted Sigma Metrics – Sigma value calculated and averaged over the observed Hb A 1c patient distribution, where σ = [(TE a -%Bias)/CV]  E(N uf ) – The number of final patient results with errors > TE a each time there is a QC out of range  In-Control % Unreliable - Probability (%) of producing results with errors > TE a when the assay is in-control

5. © Copyright 2009 by the American Association for Clinical Chemistry Introduction  Aim of this Study  To evaluate the risk of reporting an unreliable Hb A 1c result when using currently available NGSP-Certified Hb A 1c methods.  The risk assessment employed: −6 different methods across 4 academic medical centers  See Editorial:  Little, RR. Performance of Hb A 1c Assay Methods: Good Enough? Clin Chem 2014;60:8:1031.

6. © Copyright 2009 by the American Association for Clinical Chemistry Question  Besides analytical error, what other sources of error should be considered when evaluating the risk of reporting an unreliable result? How are these errors detected?

7. © Copyright 2009 by the American Association for Clinical Chemistry Materials and Methods- Hb A 1c Assays InstrumentManufacturerMethodology Variant II Turbo (version 1.0) BioRadIon-Exchange HPLC Variant IIBioRadIon-Exchange HPLC Tosoh G8TosohIon-Exchange HPLC Capillarys 2Sebia Capillary Electrophoresis Cobas Integra 800RocheImmunoassay DCA Vantage-1 (old cal) SiemensPoint of Care DCA Vantage-2 (new cal) SiemensPoint of Care

8. © Copyright 2009 by the American Association for Clinical Chemistry Materials and Methods (n=40/lab)

9. © Copyright 2009 by the American Association for Clinical Chemistry Questions  What fixed conditions were used for the assessment of risk of reporting an unreliable result for each assay?  What different TE a settings were used and why?

10. © Copyright 2009 by the American Association for Clinical Chemistry Results – Figure 2 The percentage bias of all methods was calculated using the linear regression relationships over a range of NGSP target value-assigned Hb A 1c concentrations. These biases varied widely across the different platforms with the greatest variability in the Variant II and Integra 800 assays at low Hb A 1c concentrations. Precision was determined using vendor specific QC material and ranged from 1.28% - 2.97% and 0.8% - 2.65% at the low and high levels, respectively (For more detail see table 1).

11. © Copyright 2009 by the American Association for Clinical Chemistry The predicted change in the expected number of unreliable patient results prior to an accepted QC event E(N u f), represented in the y axis and computed over a range of possible out-of control conditions [systematic error (SE)] shown on the x axis for each Hb A 1c assay platform evaluated. Each assumes a 1-2s QC rule with 2 QCs and a mean of 100 Hb A 1c examinations between QC events and an Allowable Total Error (TE a ) of 6%. The lines for the Capillary 2 (3D) and DCA Vantage Lot 2 (3G) are almost flat because the E(N uf ) is minimal. Results – Figure 3

12. © Copyright 2009 by the American Association for Clinical Chemistry Results – Table 2 Three different measures of risk of reporting an unreliable result were calculated for each assay at 3 different Allowable Total Errors (TE a = 5%, 6% and 7%) including the patient weighted sigma, the % of unreliable results while the assay is in control (In-control % unreliable), and the maximum number of unreliable results due to an out of control condition (Max E(N uf )). Each was calculated assuming a 1-2s QC rule with 2 QC events per day and 100 patient samples between QC events. The Capillarys 2 had the lowest and the Integra 800 had the highest number of expected unreliable results while the assay was in control and out of control at 5% TE a.

13. © Copyright 2009 by the American Association for Clinical Chemistry Questions  Which was more influential on method performance (precision or bias) and how can this be checked regularly?  Based on the findings of this study, Is your laboratory running an appropriate amount of QC for your Hb A 1c method?

14. © Copyright 2009 by the American Association for Clinical Chemistry Summary and Conclusions  Three types of analytical characteristics can be used to assess risk of reporting an unreliable Hb A 1c result:  Assay performance characteristics (precision and bias)  QC Strategies (# of QC samples + rules)  Clinical performance requirements (TE a goals)  Risk estimates for reporting unreliable results based upon analytical performance alone varied almost 500 fold across the 6 Hb A 1c assay platforms.  At a ±6% TE a budget, all but one Hb A 1c assay in this study requires the “max affordable” QC be run.  Risk estimates for individual laboratories’ Hb A 1c methods can be utilized to assess QC practices and residual risk for reporting an unreliable Hb A 1c result.

15. © Copyright 2009 by the American Association for Clinical Chemistry Thank you for participating in this month’s Clinical Chemistry Journal Club. Additional Journal Clubs are available at www.clinchem.org Download the free Clinical Chemistry app on iTunes for additional content! Follow us