1. Monitoring the Performance of Nucleic Acid Tests using Data Generated from EDCNet and DigitalPT Wayne Dimech, Darren Jardine, Thu-Anh Pham and the staff of the NRL

2. Presentation Outline NRL’s ever-widening role in QA Why Internet-based technology (IT) IT implications and examples for: External quality assessment schemes (EQAS) Quality control (QC) programmes Specificity monitoring Summary - effective quality monitoring

3. NRL’s Ever-widening Role in QA Monitors the performance of in vitro diagnostic (IVD) reagents for BBV in Australia including nucleic acid testing (NAT) Provides quality assurance internationally (200+ laboratories) Extending collaborations with like organisations Developing IT to support its client laboratories

4. Why Internet-based Technology Standardise data collection and storage Manage increasing data complexity Improve maintenance of data Offer better statistical analyses Give instant feedback to laboratories to ensure their ability to monitor performance Individualise reporting Offer easy administration of QA functions Offer international accessibility Facilitate international collaborations

5. NAT EQAS Design EQAS for NAT are designed to examine Limits of detection Inter- and intra-run variability Contamination/carry over Inhibition Instrument/method variability Develop QC samples Geno-subtype detection

6. IT Implications on NAT EQAS NAT EQAS concepts in DigitalPT Assay Test Process Kit Instrument Analyte Unit

7. DigitalPT Concepts Laboratory Assay Analyte Instrument Extraction Technique Kit Analyte Instrument Amplification Technique Kit Instrument Detection Technique Kit Units Kit: Roche MagNA Pure LC TNA Isolation Kit Instrument: MagNA Pure Kit: Roche COBAS Amplicor HIV-1 Monitor v 1.5 Instrument: COBAS Amplicor Kit: Roche COBAS Amplicor HIV-1 Monitor v 1.5 Instrument: COBAS Amplicor

9. IT Implications on Quality Control Collects, stores and analyses QC data Allows for inter-laboratory comparisons Requires peer groups for assessment of accuracy Can monitor imprecision and bias Can identify sources of variation

10. Quality Control Information Data collected can be used to Monitor reagent batches Troubleshoot laboratory problems Provide oversight of regional laboratories Review QC sample stability Collect information for assay standardisation Calculate uncertainty of measurement

11. Chlamydia NAT QC Programme Roche COBAS Amplicor Chlamydia QC sample produced by NRL 18 month period 15 laboratories, 28 instruments 3,570 observations; CV (%) of all results = 9.14% CV (%) ranged from 4.31 - 16.12%

12. Chlamydia NAT QC Summary QC sample, CTNG:QL:NAT:01, was tested in the Roche COBAS Amplicor Chlamydia in laboratories designated by number. The instrument, number of observations, mean and variations around the mean over the period 01/01/1999 to 21/05/2007 are shown. LabMachinenMeanSDx-2SDx+2SDMinMax%CV A COBAS 3443.660.412.844.482.604.0011.20 A COBAS 4303.860.173.524.203.363.964.40 A COBAS 5283.630.352.934.332.564.009.64 B COBAS 12663.670.223.234.112.224.005.99 B COBAS 22873.710.372.974.450.026.679.97 B COBAS 32333.820.243.344.302.724.006.28 B COBAS 42853.820.213.404.242.764.005.50 B COBAS 5283.710.163.394.033.384.004.31 B COBAS 6283.670.223.234.113.274.005.99 C COBAS 1183.520.372.784.262.984.0010.51 C COBAS 21163.720.372.984.461.834.009.95 C COBAS 31013.750.452.854.651.364.0012.00 C COBAS 4743.880.243.404.362.974.006.19 C COBAS 5323.860.233.404.322.814.005.96 18 287 Laboratory C: CV(%) COBAS 5 5.96 COBAS 4 6.19 COBAS 2 9.95 COBAS 1 10.51 COBAS 3 12.00 Mean: 3.52 to 3.86

13. Blood screening NAT QC Summary Three Chiron TMA assays monitored Period of 15 months 21 laboratories, 31 instruments Two batches of QC samples for each analyte (HBV, HCV and HIV) More than 16,000 data entries

14. Blood screening NAT QC Summary QC Sample PeliSpy Pro HCV Batch number 600403625610 AssayDuplexUltrioTIGRISDuplexUltrioTIGRIS Number2995925285295817366 Mean8.636.928.067.956.737.27 SD1.980.651.091.871.111.08 CV (%)23.09.513.623.616.514.8

15. Blood screening NAT QC Summary QC Sample PeliSpy Pro HIV Batch number 600402629201 AssayDuplexUltrioTIGRISDuplexUltrioTIGRIS Number30441222290274312765 Mean20.9812.1913.5521.0914.1615.06 SD3.881.311.972.611.221.21 CV (%)18.710.714.612.48.68.1

16. Blood screening NAT QC Summary QC Sample PeliSpy Pro HBV Batch number 600404629102 AssayUltrioTIGRISUltrioTIGRIS Number95229217265 Mean14.2315.6714.2715.01 SD1.171.041.031.41 CV (%)8.26.67.29.4

17. Chiron TMA - Duplex Results of two batches of AcroMetrix PeliSpy Pro samples tested in the Chiron Duplex TMA Assays HCV HIV A B A B QC sample batches

18. Chiron TMA - Ultrio Results of two batches of AcroMetrix PeliSpy Pro samples tested in the Chiron ULTRIO TMA Assays HCV HIV HBV A B A B A B QC sample batches

19. Chiron TMA - TIGRIS Results of two batches of AcroMetrix PeliSpy Pro samples tested in the Chiron TIGRIS TMA Assays HCV HIV HBV A B A B A B QC sample batches

20. IT Implications on Specificity Monitoring Collects and analyses the initial (IRR) and repeat reactor rates (RRR) and compares them with confirmed positive rate Monitors assay and laboratory performance Increase in IRR may indicate contamination Results may be analyzed in conjunction with QC results

21. Conclusion Internet technology allows: More complex data collection and analysis Standardisation of data collection International participation Greater scalability Facilitation of collaboration Immediate feed back to oversight bodies and manufactures on assay performance