Accreditation & Validation Joris Van Loco Scientific Institute of Public Health Food Section Joris Van Loco

Method Validation Is method validation analyzing 6 samples ? Calculating the bias, repeatability, reproducibility,… of a method ? Knowing the detection limits of the method ? knowing the uncertainty associated with a method? satisfying ISO 17025 assessors?

What is Method Validation? Method validation is the process of proving that an analytical method is acceptable for its intended purpose Joris Van Loco

Why is Method Validation Necessary? To prove what we claim is true To increase the value of test results To justify customer’s trust To trace criminals Examples To value goods for trade purposes To support health care To check the quality of drinking water Joris Van Loco

When and How should Methods be Validated New method development Revision of established methods When established methods are implemented in new laboratories Interlaboratory Comparison Single lab validation Full Validation Implementation Validation Method performance parameters are determined using equipment that is: Within specification Working correctly Adequately calibrated Competent operators Joris Van Loco

Validation and Quality Control In house validation (Bias), recovery Repeatability Within lab reproducibility Internal QC Control charts Starting data Long term within lab reproducibility Proficiency Testing Bias (trueness) Collaborative trial Reproducibility

Method Validation Accuracy Precision Selectivity (& Specificity) Trueness (CRM) Recovery (spikes) Precision Repeatability (Within) reproducibility Selectivity (& Specificity) Detection capability LOD, LOQ, CC, CC Linearity – calibration range Robustness Applicability – stability Joris Van Loco

Method Validation Performance Characteristics 2002/657/CE S: Screening methods C: Confirmatory methods Joris Van Loco

Linearity Purpose How Conclusion to evaluate the linear response of your instrument How Evaluating your calibration model Mandels fitting test Lack-of-Fit Residuals Conclusion Linear model <> other (i.e. quadratic) regression model

Linearity Residual plots (ei) Statistical tests with Lack-of-fit Mandel’s fitting test

Coefficient of correlation (r) Is NOT a suitable measure for linearity

Matrix Effect Purpose How Conclusion To evaluate whether you have a concentration dependent systematic error due the matrix i.e. ion suppression How comparison of standard curve with matrix matched standard curve Conclusion Standard solutions, spiked extracts or spiked samples for the calibration line.

Detection Limits Detection limit DIN 32645 from blanks from calibration data Funk dynamic model IUPAC Coleman recursive formula explicit formula Slide 31 There are a lot of different definitions published. This can lead to some confusion. In the next slides four different approaches discussed. Joris Van Loco

Detection Limits A) DIN 32645 Detection limit by fast estimation: Capability limit Determination limit by fast estimation Factor for fast estimation Slide 32 In the German standard DIN 32645 (partly in ISO 11843-1:1997. Capability of detection - Part 1 Terms and definitions) there are three different limits: critical value of detection , detection limit and quantitation limit (in ISO 11843 only the first and third). There are two methods described to evaluate the limits: a) from blinds; b) from calibration data. Joris Van Loco

Detection Limits B) Funk Detection limit dynamic model Determination limit dynamic model Slide 33 The approach of Funk [] is a little different. It is the intention to secure the the lower range limit. The limit of determination is comparable to the definition of DIN 32645. Joris Van Loco

Detection Limits C) IUPAC Detection limit Slide 34 The approach of the IUPAC is as ambitious as in all IUPAC-guides. Joris Van Loco

Detection limits “How to” Choose a definition and stick to it Describe the equation used in the validation file Problems statistics <> practical limitations statistics <> ID-criteria Practical LOD Analyzing samples with decreasing concentration Minimum concentration which fulfills the identification criteria = practical limit of detection Repeat the experiment S/N i.e. LOD=3xS/N

Quantitation Limit (LQ) The quantification limit is the minimum signal (concentration or amount) the can be quantified. the residual standard deviation (RSD) is included in the definition. The IUPAC default value for RSDQ= 0.1 (or 10%). LQ=10sQ.

a- and b-error a-error b-error risk of erroneously rejecting H0 i.e. risk of the conviction of an innocent b-error risk of erroneously accepting H0 i.e. risk of the non conviction of a criminal

Detection Capability Case of a permitted limit (MRL) CCa CCb +1.64sMRL +1.64ssample Signal or Concentration a = b = 5% a = 5% Joris Van Loco

Determination of CCa and CCb with ISO 11843 1,12 CCb 1,25 yc CCa CCb MRL

Detection Capability Case of a permitted limit (MRL)

Signal or Concentration Detection Capability Case of no established permitted limit or banned substance Xblank CCa CCb +2.33sblank +1.64ssample Signal or Concentration a = 1% ≠ b = 5% Joris Van Loco

Detection Capability Case of no established permitted limit or banned substance Under the assumption of linearity, normality, independence, and homoscedasticity CCa and CCb are given by: a = 2.33 b = 1.64 with b the slope of the regression line, xMRL the nominal concentration at the permitted limit; t the associated t-value, Sy the standard error of the estimate, I the number of replicates per concentration for the spiked samples; i = 1, 2, . . . , I and J the number of concentrations for the spiked samples.

Basic assumptions of the ISO 11843-2 (linear regression model) Linearity Normality Independence Error free independent variable (concentration) Homoscedasticity (↔ heteroscedasticity)

Heteroscedasticity Variance = f(x) Evaluation Impact on CCa and CCb y Homoscedasticity: s2 = constant Variance = f(x) Solution = Weighted regression Evaluation Statistical tests (cochran, Breush-pagan,…) Visual interpretation Residual plot Plot S or S² vs concentration Impact on CCa and CCb Variance at CCa and CCb is not correctly estimated CCa and CCb may be incorrect y x Heteroscedasticity: s2 = not constant

Presence of Heteroscedasticity Nitroimidazoles in plasma (MNZ-OH) Residuals plot “<“ - shape Plot of S vs conc Linear relationship between S and concentration Heteroscedasticity Impact on CCa and CCb CCa and CCb are incorrectly calculated Sblank ↓  CCa ↓ CCb ↓ or ↑

Other examples Nitroimidazoles in plasma Nitrofurans in honey Corticosteroids in liver

Weighted regression equations for CCa and CCb Solved by iteration

Conclusion detection capability Many definitions of detection limits detection limit (≈ CCa_banned substances) determination limit (≈ CCb_banned substances) Quantition limit Complicated statistics KISS demonstrate with real (spiked) samples at low concentration level  practical limit of detection

Selectivity/Specificity Identity: Signal to be attributed to the analyte GLC (change column/polarity), GC/MS, Infra-red Selectivity: The ability of the method to determine accurately the analyte of interest in the presence of other components in a sample matrix under the stated conditions of the test. Specificity is a state of perfect selectivity Joris Van Loco

Selectivity The procedure to establish selectivity: Analyze samples and reference materials Assess the ability of the methods to confirm identity and measure the analyte Choose the more appropriate method. Analyze samples Examine the effect of interferences Joris Van Loco

Selectivity: Verification of the identification criteria (2002/657/EC) MS – criteria 3 or 4 identification points 1 precursor and 2 transition ions Relative ion intensities LC – criteria Relative retention time (RRT): +/- 2.5 % (LC) UV – criteria Spectrum match +/- 3 nm CCb is concentration at or above the calculated CCb for which the ID criteria are fulfilled in 95% of the cases. CCa is concentration at or above the calculated CCa for which the ID criteria are fulfilled in 50% of the cases.

Ruggedness and Robustness Intra-laboratory study to check changes due to environmental and/or operating conditions Usually it is part of method development Deliberate changes in Temperature Reagents ( e.g. different batches) Extraction time Composition in the sample etc Joris Van Loco

Precision – ISO 5725 1-6 (1994) Expresses the closeness of agreement (dispersion level, relative standard deviation) between a series of measurements from multiple sampling of the same homogeneous sample (independent assays) under prescribed conditions. Irrespective of whether mean is a correct representation of the true value. Gives information on random errors Evaluated at three levels: repeatability intermediate precision (within laboratory) reproducibility (between laboratory)

Precision (cont.) – ISO 5725 1-6 (1994) Repeatability: precision under conditions where the results of independent assays are obtained by the same analytical procedure, on identical samples, in the same lab, by the same operator, using the same equipment and during short interval of time Intermediate precision: ISO recognizes M-factor different intermediate precision conditions (M = 1, 2 or 3) M = 1: only 1 of 3 factors (operator, equipment, time) is different M= 2 or 3: 2 or all 3 factors differ between determinations

Precision (cont.) – ISO 5725 1-6 (1994) Reproducibility: precision under conditions where results obtained: by same analytical procedure on identical sample in different laboratories, different operators, different equipment Reproducibility established by interlaboratory study (standardisation of an analytical procedure) Intermediate precision Repeatability Reproducibility

Evaluation of Precision 10 samples for each conc.under r,R, within lab R Standard Deviation Determination in pairs under r,R, within lab R Std. Dev. between two single determinations a-b, the difference between the values, d, the number of pairs sr SR SRw Joris Van Loco

Repeatability (r) and within-lab reproducibility (Rw) ANOVA table for a single factor balanced design with 3 replicate samples on the same day. Source Sum of squares df Mean Squares Expected mean squares day SSdays ndays - 1 MSdays = SSdays / (ndays – 1) σrepl² + 3σdays² replicate SSrepl nT – ndays MSrepl = SSrepl / (nT – ndays) σrepl² Total SST nT – 1 SST = MST / (nT – 1) repeatability (Sr²) and within-lab reproducibility variances (SRw²) Sr² = Srepl² SRw² = Sr² + Sdays² The Srepl²and Sdays² can be obtained from mean squares as (nrepl = 3): Srepl² = MSrepl Sdays² = (MSdays – MSrepl) / 3

Repeatability and reproducibility The value of 2.8? Variance of difference between 2 replicate measurements is 2s² Confidence interval at 95% level on the difference is 0 ± 1.96 √2 s  ± 1.96 x 1.41 sr = ± 2.8 sr  95% probability that difference between duplicate determinations will not exceed 2.8 sr r = limit of the repeatability r = 2.8 sr R = limit of the reproducibility R = 2.8 SR

Precision criteria 2002/657/CE

Horwitz: RSDR(%) = 2(1-0.5logC)

Determination of Trueness Using Certified Reference Materials Using RM or In-house materials Using Reference methods Single sample Many samples Via Interlaboratory study Joris Van Loco

Trueness, extraction yield (recovery) and apparent recovery Trueness means the closeness of agreement between the average value obtained from a large series of test results and an accepted reference value. Trueness is usually expressed as bias Recovery (extraction yield): yield of a preconcentration or extraction stage of an analytical process for an analyte divided by amount of analyte in the original sample. Apparent recovery: observed value derived from an analytical procedure by means of a calibration graph divided by reference value.

Trueness criteria 2002/657/CE When no such CRMs are available, it is acceptable that trueness of measurements is assessed through recovery of additions of known amounts of the analyte(s) to a blank matrix. Data corrected with the mean recovery are only acceptable when they fall within the ranges

Conclusions All methods must be validated (re validation might be necessary) Validation is fit for intended purpose <> determining performance characteristics accuracy profile Acceptance criteria (i.e. Horwitz) Complex statistics Relation among Validation – Quality control – proficiency testing