Observations from 20 years of PT in South Africa Dr Jim McCulloch Thistle QA Johannesburg

Who am I? Worked in labs from 1965 Experienced and qualified in Microbiology, Chemistry, Statistics and Clinical Chemistry Msc Medicine, PhD in Clinical Chemistry Setup Thistle QA in 1990

What is Thistle QA all about? Started with 1 EQA (PT) in 1990 and with 1 employee, me! Now... 23 PTs, 12 staff members 2500 labs directly enrolled Data base has results from 60 countries Data base number over 6000

Why would you want PT anyway? PICK THE CORRECT ANSWER ISO says we need it SANAS will look for it It helps achieve a quality result

Aims of PT To determine the performance of individual labs for specific tests To monitor the continuing performance of labs An essential component of a good quality system

PT Design Reference values or consensus Defined methods? Outlier detection and removal Statistics to use, robust or not? Size of data base Report issues Customer complaints

Outliers Wild results, bummers, etc Chauvenet’s Criterion Grubb’s Test Robust statistics using ISO 13528: 2005, e.g. iteration

Complaint No 1 from 1990 (We) will not be participating in cycle three of what you euphemistically call your quality control scheme. Quaint statistical manipulation and unacceptable transcription errors render your scheme more than a little suspect. Please note that we do not require reports. I would sooner the paper be saved for a more useful function than cluttering up my waste paper basket.

Bolt on extras Support Teaching material Seminars Extra samples Custom designed reports

And reports are stats They are mere sets of figures, albeit with useful information They give you warnings, not facts You can be “out of control” and not have a non-conformance Things are not always what they seem

Statistical cliché… Myth: Truth: Mark Twain said: Lies, damned lies and statistics Truth: Statistics don’t lie – it’s people who lie!

An example of wrong conclusion... 4/15/2017 Thistle QA

Don’t accept the surface

Get behind the stats 4/15/2017 Thistle QA

Why can you be okay and yet outside 2 SDs (z score >2)?

A control chart

What target value to use? By calculation from formulation CRM RM Consensus from reference labs Consensus from participating labs

The current reports 4/15/2017 Thistle QA

A Review of Chemistry PT Analytes used Sodium, measure of osmotic equilibrium Glucose, likelihood of diabetes Bilirubin, an assessment of liver function

Fitness for purpose – Acceptable Range Review of Chemistry PT Fitness for purpose – Acceptable Range Sodium 3% Glucose 7.4% Bilirubin 20.0%

Review of Chemistry PT Methodology Each lab’s results for an entire 6-month cycle Reduced to one single figure, CV, basically, SD as a percentage of the mean / average result Reviewed for changes per lab, and overall number of labs within the fitness-for-purpose percentage

Review of Chemistry PT Findings Sodium 99.9% within f-f-p range Glucose 99.3% within f-f-p Bilirubin 99.1% within f-f-p General IF first cycle out, next one always in f-f-p UNLESS instrument changes

Review of Immunoassay PT TSH, thyroid function test, assessment of thyroid stimulation T4, thyroid function output test PSA, prostate specific antigen

Review of IA (hormone testing) PT FFP TSH, 20% T4, 15% PSA, 25% Methodology As for Chemistry

Review of IA PT TSH, 94.9% within f-f-p T4, 89.9% within f-f-p PSA, 95% within f-f-p

We all see things differently 4/15/2017 Thistle QA

The real thing

Conclusions PT an essential tool Needs education, interpretation Can assist in complying with quality standard Detects changes, e.g. Instrument Cannot lead to CQI, rather compliance with f-f-p