1. Validation of screening methods (2002/657/EC)
N. Van Wouwe
IPH
AFSCA-FAVV

2. Definition (2002/657/EC) Screening method :
used to detect the presence of a substance or class of substances at the level of interest.
have the capability for a high sample throughput
=> are used to sift large numbers of samples for potential non-compliant results.
Exemple: ELISA, plate test, biosensor, receptor test,…

3. Definition (2002/657/EC)
Minimum criteria to use an analytical method as screening method:
must be validated (traceability)
must have a false compliant rate of <5% (β-error) at the level of interest

4. Performance characteristics for method validation (screening)
Qualitative method: identifies a substance on basis of its chemical, biological or physical propriety (binary response: +/-, absence/presence)
Quantitative method: determines the amount or mass fraction of a substance (response: numerical value of appropriate unit)
+ = determination is mandatory

5. Validation of screening test
Definition of the scope of the method
Analyte of group of analytes
Range of concentration
List of matrices
Initial validation with the most often used matrice in national monitoring program
Detection capacity (CCβ)
Selectivity/Specificity
Applicability/ Ruggedness/Stability
Precision (only for semi-quantitative method)
If possible: different sources of blank material, different technicians, different days on the same spiked sample

6. Validation of screening test
Targeted test: for 1 compound
validation for this compound
Targeted test: for a family of compounds
validation for 1 representative molecule of the family (antibody)
Wide range test: for more than 50 different molecules
Validation for at least a list of representative compounds
Common pattern of activity on a specific bacteria?
Common way of action (acting target)?
Published reference data on validation available?

7. Proposition of the CRL for antimicrobials (in milk)
Representative Compound
Antimicrobial class
Cefalonium/Cephapirin/Cefquinome
CEPHALOSPORINS
Penicillin G/Cloxacillin
PENICILLINS
Tetracycline/Doxcycline
TETRACYCLINS
Gentamicin/Streptomycin/Spectinomycin
AMINOGLYCOSIDES
Enrofloxacin/Flumequine
QUINOLONES
Sulfathiazole/sulfaguanidine/Sulfamerazine
SULFONAMIDES
Erythromycin/Tylosin
MACROLIDES
Lincomycin
LINCOSAMIDES
Thiamphenicol
PHENICOLATED
Trimethoprim/colistine
MISCELLANEOUS

8. Performance characteristics
Detection capacity
Selectivity/Specificity
Applicability/ Ruggedness/Stability
Precision (only for semi-quantitative method)

9. Detection capability (CCβ)
The smallest content of the substance that may be detected, identified and/or quantified in a sample with an error probability of β
In case of MRPL, CCβ= lowest concentration at which the method is able to detect truly contaminated sample with a statistical certainty of 1-β
In case of MRL, CCβ= concentration at which the method is able to detect the MRL concentrations with a statistical certainty of 1-β

10. Detection capability (CCβ)
No permitted limit
Analyse 20 blank materials => CCα = 3x signal/noise
Analyse 20 blank materials fortified at CCα
=> CCβ = CCα x SDRW
Calibration curve procedure (ISO 11843)
Analyse of blank material fortified at 0 MRLP, 0.5 MRLP, 1 MRLP, 1.5 MRLP and 2 MRLP
Plot analytical results (y-axis) vs concentration(x-axis)
CCα = y-intercept (blank) x SDRW
CCβ = CCα x SDRW

11. Detection capability (CCβ)
No permitted limit
If no quantitative results
Analyse fortified blank samples at and above CCα
(n ≥ 20 / concentration level)
CCβ = concentration level where only ≤5% false compliant results remain

12. Detection capability (CCβ)
CCa
CCb
Blank
+2.33xSDblank
+1.64xSDRW
α=1%
β=5%
Signal or Concentration

13. Detection capability (CCβ)
Permitted limit (MRL)
Analyse 20 blank materials fortified at MRL
=> CCα = MRL x SDRW
Analyse 20 blank materials fortified at CCα
=> CCβ = CCα x SDRW
Calibration curve procedure (ISO 11843)
Analyse of blank materials fortified at 0.5 MRL, 1 MRL, 1.5 MRL and 2 MRL
Plot analytical results (y-axis) vs concentration(x-axis)
CCα = MRL x SDRW
CCβ = CCα x SDRW

14. Detection capability (CCβ)
CCa
CCb
MRL
+1.64xSDMRL
+1.64xSDRW
α=5%
β=5%
Signal or Concentration

15. Performance characteristics
Detection capacity
Selectivity/Specificity
Applicability/ Ruggedness/Stability
Precision (only for semi-quantitative method)

16. Selectivity/specificity
Specificity: ability of a method to distinguish between analyte being measured and other substances
problem of interference?
F(measuring technique, class of compounds, matrices,…)

17. Selectivity/specificity
How to test specificity for qualitative screening method?
Analyse 20 different blank samples and 20 positive samples (blind study, same or different days/technicians)
Specificity= 100* NA/N-
Other parameters:
Accuracy= 100* (PA+NA)/(N- + N+)
Sensitivity= 100* PA/N+
False positive= 100* FP/(N- + N+)
False negative= 100* FN/(N- + N+)
True positive (N+)
True negative (N-)
test result positive
Positive agreement (PA)
False positive (FP)
test result negative
False negative (FN)
Negative agreement (NA)

18. Selectivity/specificity
How to test specificity for semi-quantitative screening methods?
Select potentially interfering substances (metabolites, derivatives,…)
Analyse relevant blank samples (n ≥ 20)
Analyse fortified blank samples with interfering substances at a relevant concentration
Estimate the effect of the interferences
False identification?
Influence in quantification?
Identification of the target analyte is hindered?

19. Performance characteristics
Detection capacity
Selectivity/Specificity
Applicability/ Ruggedness/Stability
Precision (only for semi-quantitative method)

20. Applicability Scope of the method must be define in term of :
Matrix (solid/liquid matrix, type of tissue)
Animal species
To introduce a new matrix
Analyse at least 10 different blank material fortified at level of interest for the new matrix (CCβ) + test of interferences
If 10 positive results => method applicable for the new matrix
If 1 negative result => 10 additional analyses
If 1 negative result=> CCβ must be recalculated for the new matrix

21. Ruggedness
Ruggedness: the susceptibility of an analytical method to changes in experimental conditions
sample material
analytes
storage condition
environmental condition
sample preparation condition

22. Ruggedness How to test ruggedness? (during development)
Identify possible factor that could influence the results (the analyst, solvents, pH, T°, rate of heating,…)
Vary each factor slightly
If one factor is found to influence results of the representative molecule, conduct further experiments
=> acceptability limits for this factor
(in the method protocol)
Recommendation of CRL: analyses of 10 blank and 10 spiked samples at the same concentration and with minor change of factor to detect influence on results

23. Stability
Test are not necessary if stability data already exist (from other lab or from publication)
To include in the validation report
Stability test:
the analyte in solution
the analyte in matrix
Aliquots of a fresh solution or sample stored under different conditions (T° and/or storing time)

24. Performance characteristics
Detection capacity
Selectivity/Specificity
Applicability/ Ruggedness/Stability
Precision (only for semi-quantitative method)

25. Precision (for quantitative screening)
Precision: the closseness of agreement between independent test results obtained under predetermined conditions
Expressed in terms of imprecision / standard deviation of test results

26. Precision (for quantitative screening)
How to test precision?
Repeatability test
within-laboratory reproducibility test(or intermediate precision)
Reproducibility test (between laboratories: interlaboratory studies)
determination of RSD (%) < Precision criteria

27. Precision (for quantitative screening)
Repeatability
3 concentrations:
1x; 1,5x; 2x MRPL
0,5; 1x; 1,5x MRL
6 replicates/level
3 times
same conditions
Within-laboratory Reproducibility
3 concentrations:
1x; 1,5x; 2x MRPL
0,5; 1x; 1,5x MRL
6 replicates/level
3 times
different conditions (analyst, env. condition,…)

28. Precision (for quantitative screening)
ANOVA treatment of data => RSDr & RSDRW
Comparison with precision criteria:
Horwitz equation: RSDR(%) = 2(1-0.5logC)
Criteria for repeatability: RSDr = 1/2 to 2/3 RSDR
Criteria for within-lab reproducibility: RSDRW = 2/3 to 1 RSDR !
For concentration < 100 µg/kg, RSDR becomes too high!

29. Other recommendations
False negative rate <5%: Analyses of 20 negative and 20 positive samples in order to test the screening method (see selectivity).
One QC sample must be added in routine and results must be added to the validation file
Method transfer/Commercial test
Bibliographical survey to compil the evaluation of performance of the test
Collection of data from supplier on validation study
Experimental plan to test skillness of technician to perform the test
Use of QC sample
Participation to proficiency test

30. Exemple: analyse of PCDD/F by CALUX bioassay
PCDD/F: 17 toxic congeners to analyse in various matrices (TCDD=most toxic dioxin)
Results expressed in TEQ (=Sum (CCixTEFi)i=1-17)
MRL for each matrix (milk, meat, egg, fish oil,…)
MRL expressed in pg TEQ/g fat or ng TEQ/ kg
Reference method: GC-HRMS
Screening method: immunoassay, bioassay,…

31. Exemple: analyse of PCDD/F by CALUX bioassay
CALUX bioassay= genetically modified cell-based bioassay (luciferase)
Amount of light produced is proportional to the toxicity (TEQ) of extracts
Gene expression
LIGHT
All substances fixing the Ah receptor

32. Analyse of PCDD/F by CALUX bioassay
Advantage:
Rapid
Cheaper than GC-HRMS
Time for analyses
Disadvantage:
Various compounds can fix the Ah receptor (PAH, PCB, PHDD/F,…)
specificity!!!!

33. Analyse of PCDD/F by CALUX bioassay : protocol
Extraction of fat
Clean-up on silica acid + carbon columns
Fraction with interfering compounds
Fraction with PCBs
Fraction with PCDD/F
Evaporation
Reading plate
Dosing plate

34. Analyse of PCDD/F by CALUX bioassay : validation
Selectivity/specificity
Ruggedness/Stability
Precision
Detection capability

35. Analyse of PCDD/F by CALUX bioassay : selectivity
Possible interfering compounds?
PAH : mostly in environmental sample
PCB: fractionation during clean-up
Other compounds? (PHDD/F): dependant of the matrix? (matrix effect?)
Results of the selectivity test:
No interferences for feedstuff, milk, egg, fat
Interferences for fish oil
CALUX results = 2 x GC-HRMS results

36. Analyse of PCDD/F by CALUX bioassay : selectivity
Matrix effect for fish oil

37. Analyse of PCDD/F by CALUX bioassay : ruggedness
What are the critical point in the protocol?
Carbon column (interferences)
Solvent (interferences)
Curve (results)
Evaporation time (recovery)
Age of CALUX cell line (RSD)

38. Analyse of PCDD/F by CALUX bioassay : ruggedness
Carbon column: amount of carbon used
(Rdt PCDD/F= 60%)
(Rdt PCDD/F= 80%)
DX fraction
PCB fraction
Not collected fraction
(Rdt PCDD/F= 80%)

39. Analyse of PCDD/F by CALUX bioassay : ruggedness
Evaporation time

40. Analyse of PCDD/F by CALUX bioassay : ruggedness
Solvent: tested before use on a TCDD solution (antagonist/agonist effect)
Curve: tested with an independant TCDD solution
Age of CALUX cells: new cell every 2 months

41. Analyse of PCDD/F by CALUX bioassay : precision
Validation protocol
Day 1
Day 2
Day 3
Day 4
Day 5
Day 6
Blank solvent 1
Blank sample
Sample at MRL/2 3 2
Sample at MRL 6
Sample at 2MRL
Quality sample 10

42. Analyse of PCDD/F by CALUX bioassay : precision
ANOVA results for the TEQ determination of PCDD/F in feedstuff by CALUX bioassay
At MRL (0.75ng TEQ/kg) : XMRL= ng TEQ/kg
Sr= => RSDr= 8.4%
SRW= =>RSDRW= 9.7%
At MRL/2 (0.376ng TEQ/kg) : XMRL/2= ng TEQ/kg
Sr= => RSDr= 11%
SRW= =>RSDRW= 11%
At 2MRL (1.5ng TEQ/kg): X2MRL= ng TEQ/kg
Sr= => RSDr= 6.8%
SRW= =>RSDRW= 7.3%
RSD < 30%
(2002/70/EC)

43. Analyse of PCDD/F by CALUX bioassay : detection capacity
CCβ for the TEQ determination of PCDD/F in feedstuff by CALUX bioassay
CCα = MRL x SRW
CCα = x = 0.87 ng TEQ/kg
CCβ = CCα x SRW
CCβ = x = 1.04 ng TEQ/kg
2002/70/EC: false negative rate < 1% !
=> At a concentration of 1.04ng TEQ/kg, we are sure that the sample is a positive sample with 99% certainty

44. Analyse of PCDD/F by CALUX bioassay :confirmatory range?
COMPLIANT
SUSPICIOUS
NON COMPLIANT
CC*
MRL
CCa
CCb
-2.33sMRL
+1.64sMRL
+2.33ssample
Signal or Concentration
* =1%
β=5%
α=5%

45. Analyse of PCDD/F by CALUX bioassay :confirmatory range
Lower limit of the confirmatory range for the TEQ determination of PCDD/F in feedstuff by CALUX bioassay
CC*= MRL-2.33 x SDRW
CC*= x = 0.58 ng TEQ/kg
Conclusion
Sample lower than 0.58 ng TEQ/kg are negative with 99% certainty (false negative rate < 1%)
Sample above 0.58 ng TEQ/kg must be confirmed by GC-HRMS