1. Accreditation & Validation
Joris Van Loco
Scientific Institute of Public Health
Food Section
Joris Van Loco

2. Method Validation Is method validation analyzing 6 samples ?
Calculating the bias, repeatability, reproducibility,… of a method ?
Knowing the detection limits of the method ?
knowing the uncertainty associated with a method?
satisfying ISO assessors?

3. What is Method Validation?
Method validation is the process of proving that an analytical method is acceptable for its intended purpose
Joris Van Loco

4. Why is Method Validation Necessary?
To prove what we claim is true
To increase the value of test results
To justify customer’s trust
To trace criminals
Examples
To value goods for trade purposes
To support health care
To check the quality of drinking water
Joris Van Loco

5. When and How should Methods be Validated
New method development
Revision of established methods
When established methods are implemented in new laboratories
Interlaboratory Comparison
Single lab validation
Full Validation
Implementation Validation
Method performance parameters are determined using
equipment that is:
Within specification
Working correctly
Adequately calibrated
Competent operators
Joris Van Loco

6. Validation and Quality Control
In house validation
(Bias), recovery
Repeatability
Within lab reproducibility
Internal QC
Control charts
Starting data
Long term within lab reproducibility
Proficiency Testing
Bias (trueness)
Collaborative trial
Reproducibility

7. Method Validation Accuracy Precision Selectivity (& Specificity)
Trueness (CRM)
Recovery (spikes)
Precision
Repeatability
(Within) reproducibility
Selectivity (& Specificity)
Detection capability
LOD, LOQ, CC, CC
Linearity – calibration range
Robustness
Applicability – stability
Joris Van Loco

8. Method Validation Performance Characteristics 2002/657/CE
S: Screening methods
C: Confirmatory methods
Joris Van Loco

9. Linearity Purpose How Conclusion
to evaluate the linear response of your instrument
How
Evaluating your calibration model
Mandels fitting test
Lack-of-Fit
Residuals
Conclusion
Linear model
<> other (i.e. quadratic) regression model

10. Linearity Residual plots (ei) Statistical tests with Lack-of-fit
Mandel’s fitting test

11. Coefficient of correlation (r)
Is NOT a suitable measure for linearity

12. Matrix Effect Purpose How Conclusion
To evaluate whether you have a concentration dependent systematic error due the matrix
i.e. ion suppression
How
comparison of standard curve with matrix matched standard curve
Conclusion
Standard solutions, spiked extracts or spiked samples for the calibration line.

13. Detection Limits Detection limit DIN 32645 from blanks
from calibration data
Funk dynamic model
IUPAC
Coleman recursive formula
explicit formula
Slide 31
There are a lot of different definitions published. This can lead to some confusion.
In the next slides four different approaches discussed.
Joris Van Loco

14. Detection Limits A) DIN 32645 Detection limit by fast estimation:
Capability limit
Determination limit
by fast estimation
Factor for fast estimation
Slide 32
In the German standard DIN (partly in ISO :1997. Capability of detection - Part 1 Terms and definitions) there are three different limits: critical value of detection , detection limit and quantitation limit (in ISO only the first and third). There are two methods described to evaluate the limits: a) from blinds; b) from calibration data.
Joris Van Loco

15. Detection Limits B) Funk Detection limit dynamic model
Determination limit dynamic model
Slide 33
The approach of Funk [] is a little different. It is the intention to secure the the lower range limit. The limit of determination is comparable to the definition of DIN
Joris Van Loco

16. Detection Limits C) IUPAC Detection limit Slide 34
The approach of the IUPAC is as ambitious as in all IUPAC-guides.
Joris Van Loco

17. Detection limits “How to”
Choose a definition and stick to it
Describe the equation used in the validation file
Problems
statistics <> practical limitations
statistics <> ID-criteria
Practical LOD
Analyzing samples with decreasing concentration
Minimum concentration which fulfills the identification criteria = practical limit of detection
Repeat the experiment
S/N
i.e. LOD=3xS/N

18. Quantitation Limit (LQ)
The quantification limit is the minimum signal (concentration or amount) the can be quantified.
the residual standard deviation (RSD) is included in the definition.
The IUPAC default value for RSDQ= 0.1 (or 10%). LQ=10sQ.

19. a- and b-error a-error b-error risk of erroneously rejecting H0
i.e. risk of the conviction of an innocent
b-error
risk of erroneously accepting H0
i.e. risk of the non conviction of a criminal

20. Detection Capability Case of a permitted limit (MRL)
CCa
CCb
+1.64sMRL
+1.64ssample
Signal or Concentration
a = b = 5%
a = 5%
Joris Van Loco

21. Determination of CCa and CCb with ISO 11843
1,12
CCb
1,25 yc
CCa
CCb
MRL

22. Detection Capability Case of a permitted limit (MRL)

23. Signal or Concentration
Detection Capability Case of no established permitted limit or banned substance
Xblank
CCa
CCb
+2.33sblank
+1.64ssample
Signal or Concentration
a = 1% ≠ b = 5%
Joris Van Loco

24. Detection Capability Case of no established permitted limit or banned substance
Under the assumption of linearity, normality, independence, and homoscedasticity CCa and CCb are given by:
a = 2.33
b = 1.64
with b the slope of the regression line, xMRL the nominal concentration at the permitted limit; t the associated t-value, Sy the standard error of the estimate, I the number of replicates per concentration for the spiked samples; i = 1, 2, , I and J the number of concentrations for the spiked samples.

25. Basic assumptions of the ISO 11843-2 (linear regression model)
Linearity
Normality
Independence
Error free independent variable (concentration)
Homoscedasticity (↔ heteroscedasticity)

26. Heteroscedasticity Variance = f(x) Evaluation Impact on CCa and CCb y
Homoscedasticity:
s2 = constant
Variance = f(x)
Solution = Weighted regression
Evaluation
Statistical tests (cochran, Breush-pagan,…)
Visual interpretation
Residual plot
Plot S or S² vs concentration
Impact on CCa and CCb
Variance at CCa and CCb is not correctly estimated
CCa and CCb may be incorrect y x
Heteroscedasticity:
s2 = not constant

27. Presence of Heteroscedasticity
Nitroimidazoles in plasma (MNZ-OH)
Residuals plot
“<“ - shape
Plot of S vs conc
Linear relationship between S and concentration
Heteroscedasticity
Impact on CCa and CCb
CCa and CCb are incorrectly calculated
Sblank ↓  CCa ↓
CCb ↓ or ↑

28. Other examples Nitroimidazoles in plasma Nitrofurans in honey
Corticosteroids in liver

29. Weighted regression equations for CCa and CCb
Solved by iteration

30. Conclusion detection capability
Many definitions of detection limits
detection limit (≈ CCa_banned substances)
determination limit (≈ CCb_banned substances)
Quantition limit
Complicated statistics
KISS
demonstrate with real (spiked) samples at low concentration level  practical limit of detection

31. Selectivity/Specificity
Identity: Signal to be attributed to the analyte
GLC (change column/polarity), GC/MS, Infra-red
Selectivity: The ability of the method to determine accurately the analyte of interest in the presence of other components in a sample matrix under the stated conditions of the test.
Specificity is a state of perfect selectivity
Joris Van Loco

32. Selectivity The procedure to establish selectivity:
Analyze samples and reference materials
Assess the ability of the methods to confirm identity and measure the analyte
Choose the more appropriate method.
Analyze samples
Examine the effect of interferences
Joris Van Loco

33. Selectivity: Verification of the identification criteria (2002/657/EC)
MS – criteria
3 or 4 identification points
1 precursor and 2 transition ions
Relative ion intensities
LC – criteria
Relative retention time (RRT): +/- 2.5 % (LC)
UV – criteria
Spectrum match
+/- 3 nm
CCb is concentration at or above the calculated CCb for which the ID criteria are fulfilled in 95% of the cases.
CCa is concentration at or above the calculated CCa for which the ID criteria are fulfilled in 50% of the cases.

34. Ruggedness and Robustness
Intra-laboratory study to check changes due to environmental and/or operating conditions
Usually it is part of method development
Deliberate changes in
Temperature
Reagents ( e.g. different batches)
Extraction time
Composition in the sample
etc
Joris Van Loco

35. Precision – ISO (1994)
Expresses the closeness of agreement (dispersion level, relative standard deviation) between a series of measurements from multiple sampling of the same homogeneous sample (independent assays) under prescribed conditions.
Irrespective of whether mean is a correct representation of the true value.
Gives information on random errors
Evaluated at three levels:
repeatability
intermediate precision (within laboratory)
reproducibility (between laboratory)

36. Precision (cont.) – ISO 5725 1-6 (1994)
Repeatability: precision under conditions where the results of independent assays are obtained by the same analytical procedure, on identical samples, in the same lab, by the same operator, using the same equipment and during short interval of time
Intermediate precision: ISO recognizes M-factor different intermediate precision conditions (M = 1, 2 or 3)
M = 1: only 1 of 3 factors (operator, equipment, time) is different
M= 2 or 3: 2 or all 3 factors differ between determinations

37. Precision (cont.) – ISO 5725 1-6 (1994)
Reproducibility: precision under conditions where results obtained:
by same analytical procedure
on identical sample
in different laboratories, different operators, different equipment
Reproducibility established by interlaboratory study (standardisation of an analytical procedure)
Intermediate precision
Repeatability Reproducibility

38. Evaluation of Precision
10 samples for each conc.under r,R, within lab R
Standard Deviation
Determination in pairs under r,R, within lab R
Std. Dev. between two single determinations
a-b, the difference between the values, d, the number of pairs sr SR
SRw
Joris Van Loco

39. Repeatability (r) and within-lab reproducibility (Rw)
ANOVA table for a single factor balanced design with 3 replicate samples on the same day.
Source
Sum of squares df
Mean Squares
Expected mean squares
day
SSdays
ndays - 1
MSdays = SSdays / (ndays – 1)
σrepl² + 3σdays²
replicate
SSrepl
nT – ndays
MSrepl = SSrepl / (nT – ndays)
σrepl²
Total
SST
nT – 1
SST = MST / (nT – 1)
repeatability (Sr²) and within-lab reproducibility variances (SRw²)
Sr² = Srepl²
SRw² = Sr² + Sdays²
The Srepl²and Sdays² can be obtained from mean squares as (nrepl = 3):
Srepl² = MSrepl
Sdays² = (MSdays – MSrepl) / 3

40. Repeatability and reproducibility
The value of 2.8?
Variance of difference between 2 replicate measurements is 2s²
Confidence interval at 95% level on the difference is 0 ± 1.96 √2 s  ± 1.96 x 1.41 sr = ± 2.8 sr
 95% probability that difference between duplicate determinations will not exceed 2.8 sr
r = limit of the repeatability r = 2.8 sr
R = limit of the reproducibility R = 2.8 SR

41. Precision criteria 2002/657/CE

42. Horwitz: RSDR(%) = 2(1-0.5logC)

43. Determination of Trueness
Using Certified Reference Materials
Using RM or In-house materials
Using Reference methods
Single sample
Many samples
Via Interlaboratory study
Joris Van Loco

44. Trueness, extraction yield (recovery) and apparent recovery
Trueness means the closeness of agreement between the average value obtained from a large series of test results and an accepted reference value. Trueness is usually expressed as bias
Recovery (extraction yield): yield of a preconcentration or extraction stage of an analytical process for an analyte divided by amount of analyte in the original sample.
Apparent recovery: observed value derived from an analytical procedure by means of a calibration graph divided by reference value.

45. Trueness criteria 2002/657/CE
When no such CRMs are available, it is acceptable that trueness of measurements is assessed through recovery of additions of known amounts of the analyte(s) to a blank matrix. Data corrected with the mean recovery are only acceptable when they fall within the ranges

46. Conclusions
All methods must be validated (re validation might be necessary)
Validation is fit for intended purpose <> determining performance characteristics
accuracy profile
Acceptance criteria (i.e. Horwitz)
Complex statistics
Relation among
Validation – Quality control – proficiency testing