1. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 1 A Survey of Water Quality Comparison of on-site test kits and laboratory analytical methods when performing a survey of water quality balancing the three factors of time, cost and quality. A successful survey requires valid analytical measurements.

2. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 2 Six Principles for Valid Analytical Measurement Agreed analytical requirement Methods/equipment fit-for-purpose Competent staff Regular checks on technical performance Consistency of measurements Quality Assurance/Quality Control

3. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 3 The Scenario You have received a phone call from a client asking if you can perform a water quality survey.

4. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 4 Possible Client Requirements Why does the client require the survey? Regulatory requirement (e.g. drinking water quality fit for human consumption) Monitoring (e.g. survey for corrosion within the hot water supply) Identification (e.g. people taken ill in a swimming pool) Dispute (e.g. is there benzene in mineral water)

5. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 5 The Student’s Assignment The assignment is to : perform a survey to satisfy the client’s requirements. compare the performance of test kits and laboratory methods in terms of time, cost and quality. learn how the VAM Principles provide a framework to achieve this objective.

6. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 6 Comparing the Performance of Methods: e.g. pH Client’s requirement Test kit method Laboratory method Universal pH paper rangeCalibrated pH 0-14 in steps of 0.5 electrode Swimming pool water pH range: 7.2 and 7.8trueness (bias)? ideally 7.4 and 7.6 cost? Drinking water supply pH range:5.5 and 9.5 ideally 6.5 and 8.5 precision?

7. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 7 Performance in Terms of Quality Are the test kits/lab methods sufficiently accurate? Are the test kits/lab methods sufficiently precise? Are the test kits/lab methods under control?

8. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 8 Method Validation - Parameters Method validation might include: Selectivity and specificity Range Linearity Limit of detection and limit of quantitation Ruggedness Precision and bias (uncertainty)

9. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 9 Performance in Terms of Cost and Time Can the client afford the cost of the analysis? Can the laboratory produce the results in time? Is the test kit/lab method ‘fit for purpose’?

10. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 10 Competent Personnel - Staff Validation? People who make analytical measurements should be: Qualified Trained Motivated Rewarded financially

11. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 11 Precision For precision measurements you will need: a large homogenous sample. the analyte to be present in the sample matrix at an appropriate concentration. to perform at least six repeat measurements. to perform a statistical analysis (mean, standard deviation, coefficient of variation and 95% confidence limits). to use the statistical analysis to make a judgement.

12. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 12 The Role of Quality Control Charts Target Measured value Upper action Lower action Upper warning limit Lower warning limit

13. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 13 Bias For bias measurements you will need: a homogenous sample containing a known concentration of analyte (e.g. a reference material). to perform at least three repeat measurements. to perform a statistical analysis (mean, standard deviation). to use the statistical results to make a judgement about bias.

14. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 14 Are the Methods / Test Kits “Fit for Purpose” Are the methods accurate and precise enough for their intended use? Do the methods perform reliably (i.e. reproducible results)? Do the methods perform satisfactorily (e.g. limit of detection)? Are they cost effective? Can they produce the results in time?

15. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 15 Sampling Strategy Use the client’s requirements to: determine the number of samples to be taken. identify where are the samples to be taken from. decide what size of sample is required. decide if the sample needs special storage conditions.

16. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 16 Sample Information Records Purpose for taking sample Identification numbers & traceability of samples (trackability) Sample location and date of sampling Sampling appearance Observations

17. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 17 Preparing for the Sampling Operation Carry out a trial run Think about safety (e.g. carrying the sampling equipment). Identify and obtain appropriate sampling equipment. Make a list of everything you require to complete the survey. Plan how you are going to maintain sample integrity.

18. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 18 Undertaking the Survey The survey will be a success if you have: devised a sampling strategy. created a system for recording sample information and on site test results. made suitable preparations for the sampling operation. learnt how to use the on site test kits. made arrangements to preserve the integrity of the samples.

19. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 19 Undertaking the Analysis The survey will be a success if you have: checked the methods perform satisfactorily. been trained in the use of the methods. in place a quality control scheme for regular monitoring. relevant sample information to link to the analytical results.

20. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 20 Writing the Report There are two types of report that can be written depending on the client’s requirements: A report to a client detailing the findings of the survey regarding the quality of water. A technical report to an expert regarding the use of test kits in terms of cost, time and quality.

21. B. Neidhart, W. Wegscheider (Eds.): Quality in Chemical Measurements © Springer-Verlag Berlin Heidelberg 2000 P. HoulgateAssessment of Test Kits in Terms of Time, Cost and Quality 21 The Role of VAM in Performing the Survey: Agreed analytical requirement Methods/equipment fit-for-purpose Competent staff Consistency of measurements Quality Assurance/Quality Control VAM principles apply to all aspects of the survey