1. General Strategy for Analytical Quality Control Design and analysis of quality control data Pentti Minkkinen Professor emeritus (LUT) E-mail: Pentti.Minkkinen@lut.fiPentti.Minkkinen@lut.fi CONSULTING E-mail: pentti.minkkinen@sirpeka.fipentti.minkkinen@sirpeka.fi Copyright by Pentti Minkkinen Quality Control of Chemical and Environmental Measurements: Concepts and Methods, Kuopio, 12-15-1-2008

2. Introduction The errors of analytical measurements, random as well as systematic errors nearly always are dependent on concen- tration, especially if the same method is used over a wide concentration range. Tests based on normal distribution are, therefore, seldom applicable to the original analytical quality control data; some data preprocessing is usually necessary. The purpose of this presentation is to introduce methods which take the concentration dependence into account.

3.   x LOT Primary Secondary Analysis Result sample s 1 s 2 s 3 s x Propagation of errors: Example: GOAL: x =  Analytical process usually contains several sampling and sample preparation steps

4. Estimation of the uncertainty of the results generated by random errors Estimation of the random errors cannot be based on reference materials, because they do not pass all the steps of the total measurement process (e.g., primary sampling) reference materials are carefully prepared and homogenized differences in matrices and concentration levels THE UNCERTAINTY ESTIMATES HAVE TO BE BASED ON MEASUREMENTS CARRIED ON ROUTINE SAMPLES

5. Theory of sampling (by P. Gy) Gives guidance –For designing sampling equipment and procedures –For auditing and monitoring of sampling protocols –For designing sampling experiments –For designing cost effective fit-for-purpose sampling protocols

6. Error components of analytical determination according to P. Gy Global Estimation Error GEE Global Estimation Error GEE Total Sampling Error TSE Point Selection Error PSE Total Analytical Error TAE Point Materialization Error PME Point Materialization Error PME Weighting Error SWE Increment Delimi- tation Error IDE Increment Delimi- tation Error IDE Long Range Point Selection Error PSE 1 Periodic Point Selection Error PSE 2 Fundamental Sampling Error FSE Grouping and Segregation Error GSE Increment Extraction Error IEE Increment Extraction Error IEE Increment and Sample Preparation Error IPE Increment and Sample Preparation Error IPE GEE=TSE +TAE TSE= (PSE+FSE+GSE)+(IDE+IXE+IPE)+SWE

7. Sample materialization error Audit procedures Minimized by using equipment designed and operated according to the rules of correct sampling Statistical errors Contribution of the statistical errors to the total uncertainty of the measurement can be estimated only experimentally (FSE exception, if material properties are known)

8. Uncertainty generated by random variation (repeatability, or precision of the measurements) can always be estimated from repeated measurements. From the replicates the standard deviation (or standard uncertainty) is estimated. Replicates have to carried out at least at two levels: 1. From a part of the laboratory samples parallel analytical samples are made and analyzed  this gives the uncertainty estimate of the laboratory method 2. At the selected field locations parallel replicates of the field samples are taken and analyzed independently  this data provides the uncertainty estimate of the complete measurement chain EXPERIMENTAL DESIGN Make only two, maximum three replicates at each selected field site (or laboratory sample), but make replicates at as many field locations and laboratory sample as the budget allows, e.g. 5-10 % of the number of the routine samples.

9. MODEL (TOTAL MEASUREMENT VARIANCE): within-laboratory variance Variance of primary sample (short- range sampling variance) between-sampling-sites variance (or long-range variance) (4)(4)

10. x 111 x 112 x 122 x 211 x 212 x 222 x 121 x 221 x n11 x n12 x n21 x n22 1 2 n Sampling sites (field replicates) s22s22 s32s32 Calculation of three independent variance estimates from analytical results 2xs 2xs xs2 1 2xs 2 2s n xs2 1 2 n s12s12 Basic design for experimental estimation of sampling variances Splitting and analysis of field samples (laboratory replicates)

11. Notations Number of field sampling sites: n, indices of analytical results i = 1, 2,…, n Number of replicate determinations per each laboratory sample: l, indices k =1, 2,…, l (l = 2 in Fig. 1) Number of field replicates per site: r, indices of results, j=1, 2,…, r (r = 2 in Fig.1) Analytical results: x ijk

12. Calculation of independent variance estimates 1. Standard deviation and mean for each lab. sample: 2. Calculate from the mean values,, the standard deviation and the mean of each field location: 3. From the previous results three independent variance estimates can be calculated:, with degrees of freedom, (1), with degrees of freedom, (2)

13. If the mean results of the sampling sites are not autocorrelated s3 is calculated If the mean results of the sampling sites are autocorrelated, Gy’s variographic technique can be used to estimate the point selection error (PSE)., with degrees of freedom, (3)

14. MODEL (TOTAL MEASUREMENT VARIANCE): within-laboratory variance Variance of primary sample (short- range sampling variance) between-sampling-sites variance (or long-range variance) (4)(4)

15. l = number of laboratory replicates per field sample r = number of field replicates per site The variance components obtained from the experiment are linear combinations of the underlying components of the measurement variances (5)(5) (6)(6) (7)(7)

16. Significance tests, degrees of freedom ( ) (8) If this test is significant at the selected confidence level we can conclude that and its estimate is obtained as (9) If the test F 1 is not significant we can accept the hypothesis

17. If this test is significant  b can be estimated as (11) degrees of freedom ( ) (10)

18. In case neither of the tests F 1 and F 2 is significant a third F-test can be calculated degrees of freedom ( ) (12) If F 3 is significant, it means that the sum of the short-range and long-range variances is significantly different from zero, but they cannot be estimated reliably by using this experimental data

19. How to use the results? 1. Uncertainty of a single sample a) Result is used as an estimate for the mean of the sampling site (13), the starting value of variogram

20. b) Result is used as an estimate for the mean of the whole lot (14)

21. 2. Uncertainty of the mean of several samples a) Result is used as an estimate for the mean of the sampling site (14) b) Mean of a single sampling site is used as an estimate for the mean of the whole lot (15)

22. c) Mean of the lot is based on several (=n) sampling sites (15) When the variance estimates and cost estimates of the different steps of the measurement process are known an optimized fit-for- purpose sampling and measurement protocol can be designed. In optimization either  Budget is given and total variance is minimized  Tolerable variance (uncertainty) is defined and the budget is minimized

23. EXAMPLES Uncertainty of water sampling (lake, river and wastewater treatment facilities) for environmental monitoring was estimated by using the proposed design

24. pH Measurement

25. Results of ANOVA: pH measurement F-test p 13.83 1.00 1.83 0.98* s1 2 s2 2 1 2 0.00088 0.00610 68 34 0.00091 0.00083 66 33* s1 s s s t 1 s t 0.0297 0.0752 0.081 68 29.2 39 0.0302 0.0195 0.036 66 5.9 68.6* *Highest point rejected

26. AMMONIUM-NITROGEN

27. Estimation of the sampling errors of NH 4 -N determination in water samples. ANOVA is based on relative standard deviations (standard deviation divided by Site mean).

28. Results: NH 4 -N Relative Standard Degrees of deviation freedom sr 1 = 3.4 % 1 = 10 sr s = 3.8 % s = 2.5 sr t = 5.1 % t = 7.9

29. Example 2: sampling error estimation for dissolved oxygen in water

30. Dissolved oxygen in Water samples: ANOVA based on absolute variances

31. RESULTS: Dissolved Oxygen Standard deviation Degrees of freedom s1 = 0.094 mg/L n1 =20 ss = 0.21 mg/L n1 =8.2 st = 0.23 mg/L n1 =11.9

32. TABLE 1: ANOVA based on relative standard deviations Analyte sr1 % 1 sr s % s sr t % t Concentration range COD_Mn3.1603.2134.450.1> 3 mg/l Na1.5281.982.420.7> 3 mg/l SS7.124117.91316.4> 0.5 mg/l Tot.-N4.3542.53.6 58.8 > 350  g/l NH 4 -N3.4103.82.55.27.9 > 10  g/l NO 3 -N1.5105.54.75.75.4 > 20  g/l Turbidity5.6601123.31237.1> 1 FTU Conduct.1.4662.322.82.744> 5 mS/m Color6.6648.117.51048.5> 5 mg Pt/l Tot.-P5.2661328.11438.1 > 4  g/l

33. Table 2: ANOVA based on absolute standard deviations Analyte s1 1s s sr t t Concentration range PO 4 -P0.682.73.82.74.2 < 75  g/l pH0.030660.0205.90.03668.66-9 pH-units O 2 -conc.0.078540.1621.70.1832.6< 13 mg/l O 2 -satur.1.2541.414.11.841.7< 110 % Chlorophyll0.078100.1640.186 < 5  g/l

34. 2 3 4 5 6 7 1 n 8 Splitting and analysis of field samples (laboratory duplicates) Sampling sites (field duplicates) x 311 x 511 x 611 x 811 x 111 x 121 x n11 x n21 sm2 1 sm2 n x 211 x 212 sm1 21 x 411 x 412 sm1 41 x 711 x 712 sm1 71 Simplified design for monitoring the sampling variances

35. If in the monitoring campaign the No. of field sites with duplicate samples is n m and the No. of laboratory samples is l m, the degrees of freedom sm1 and sm2, respectively, are

36. Stability of the sampling and measurement process can now be estimated by comparing the monitoring variances to previous variances by using appropriate F-test Monitoring results can be visualized as Control Chart

37. Estimation of the uncertainty of a laboratory or an analytical methods Uncertainty from random errors –Absolute or relative standard deviation does not depend on concentration –Absolute or relative standard deviation are functions of concentration Systematic errors

38. Examples Estimation of the uncertainty of the laboratory measurements from the replicates. Whether or not the concentration dependence has to be taken into account can often be judged just by plotting either the absolute or relative standard deviations of the replicates vs. concentration.

39. Absolute standard deviation can be regarded independent on concentration Oxygen content (mg/l) of water samples. Uncertainty of the laboratory measurement Fig. 1: Standard deviations of the replicates do not show a concentration dependence within the range studied 02468101214 0 0.05 0.1 0.15 0.2 0.25 Concentration (mg/l) s i mg/l) Expanded uncertainty:

40. Relative standard deviation independent on concentration The concentration dependence of the absolute standard deviation, if present, can often be modeled with either one of the following simple equations: (1) (2) s c = standard deviation in concentration units, s 0 = standard deviation of the blank, c = concentration and b = slope

41. If the absolute standard deviations based on eqs. 1 and 2 are converted to relative standard deviations the following results are obtained: (3) (4) When concentration increases according to both equations the relative standard deviation approaches asymptotically a constant value (Fig. 2) At the optimal concentration range the relative standard deviation of the method can regarded independent on concen- tration and the and the individual estimates can be pooled.

42. 020406080100 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 Concentration Relative standard deviation Fig.2: According to both models, 3 (red) and 4 (blue), the relative standard deviation quickly approaches a constant value when the concentration increases from the detection limit. If the method is used at this optimum level the relative standard deviation of the replicates can be pooled (Eq. 5).

43. If, at the concentration range studied the relative standard deviation can be regarded constant the RSD estimates of the replicates can be pooled: (5) If the number of samples is n and on each r replicated are made the degrees of freedom of s r are (6)

44. Uncertainty of the AAS laboratory method used for the determination of lead from geological samples 020004000600080001000012000140001600018000 0 5 10 15 20 25 Concentration (ppm) s r (%) Pooled relative standard deviation = 7,6 % Expanded uncertainty:

45. Both the absolute and relative standard deviation depend on concentration If neither absolute nor relative standard deviation can be regarded constant the concentration dependence has to be modeled. One can first check if Equations1-4 fit to the data sufficiently well. These equations can be linearized as shown in Fig. 3 To fit the data to the linearized models linear regression can be used

46. Equations 1-4 linearized c scsc (1) c2c2 sc2sc2 (2) 1/c 2 sr2sr2 (4) 1/c srsr (3)

47. If the absolute or relative standard deviations are modeled according to Eqs. 1 or 3 the result is equivalent of the mean of the standard deviation estimates (not the pooled standard deviation) and the result has to be corrected by multiplying with the following correction factor, k, which depends on the degrees of freedom of the standard deviation of the replicates.

48. EXAMPLE DATA: Journal of AOAC International 84 (2001) 226- 235. The method was characterized by carrying 5 replicate measurements at 7 concentration levels. Both s and s r are dependent on concen- tration. All four models fit the data fairly well but the fit to Eq.1gave the most significant regression coefficients

49. A B A.Eq. 1 fitted to the data B.Model 1 converted to relative standard deviation, s r =s/c, and tested with experimental data

50. Results: Regression: Correction factor (  = 4): k=1,06 Corrected equation: Dependence of the relative standard deviation on concentration: The standard deviation estimates are based on seven concen- tration levels and thus have 7-2 = 5 degrees of freedom

51. Different decision limits used in analytical chemistry can now be defined with the help of these results, e.g.: Limit of Determination: concentration at which the expanded relative uncertainty U=t (,2  =0.05)· s r reaches the required level, e.g., 50 %. Similarly, the limit of determination, c QL is Limit of Detection: concentration at which the expanded relative uncertainty U=t (,2  =0.05)· s r = 100 %. By substituting U = 1 and t (,2  =0.05) = 2.571 detection limit, c DL can be solved from the equation of s r :

52. Systematic errors and method comparisons Systematic measurement errors have many different causes, e.g., Calibration Matrix effects Incorrect sampling Contamination

53. Control of Systematic Errors Analysis of blank samples (contamination control) Analysis of reference materials within batches of routine samples shows, e.g. when the calibration needs updating. Results are often presented as control charts

54. Analysis of known additions (recovery tests) Comparisons by using different, indepen- dent analytical methods. The methods to be independent requires, e.g., that calibration are carried out independently. The results of a process analyzer can be compared to laboratory results.

55. Interlaboratory comparisons either by using the same or different methods. This method is often used to produce certified reference materials and in proficiency testing. Plausibility check by an expert. Obvious errors (strange combination of results) can often be detected by using this method.

56. Comparison of test results with reference values To check the possible concentration dependence of the errors the data analysis is started either by plotting the differences of the methods (Eq. 7) or relative differences (Eq. 8) vs. concentration (reference values) (7) (8) x are the test results and x ref either the known concentrations of the reference materials or the results of the reference method. Typical shapes of the graphs are shown below. Based on the information obtained from the graphs the data analysis is continued

57. B4 c drdr B3 c drdr B2 c drdr B1 c drdr A4 c d A3 c d A2 c d A1 c d Typical shapes of the absolute (A) and relative (B) differences of the test method from the reference values

58. Data analysis A1 and B1: In A1 both the absolute and in B1 the relative random and and systematic errors are independent on concentration at the studied concentration range. The mean of d (or d r ) is the best estimate of the systematic difference between the methods. A1 B1 c c drdr d

59. or (9) (10) If zero is included in the confidence interval the possible systematic error is not significantly different from zero. Confidence intervals:

60. A2 and B2: Random error (absolute A, relative B) depends on concentration but systematic error is constant. The distribution is not normal and, consequently, t-test or the equivalent confidence interval calculation is not applicable in testing the significance of the systematic error. Some robust test, e.g., the sign test can be used to test the presence of the systematic error. A2 B2 c c drdr d

61. However, if the error structure is known the differences can be weighted to normalize the results and t- test can be applied. The mean or weighted mean of d:n (or d r ) is again the best estimate of the systematic difference between the test method and the reference values.

62. A3 ja B3: Random error is independent on concentration but system- atic error has a functional dependence on concentration. In this case d or d r should be modeled as a function of concen- tration. Usually some data pretreatment is needed to linearize the the dependence on concentration. A3 B3 c c drdr d

63. A4 ja B4: Both the random and systematic errors are functions of concentration. To model the d or d r weighted regression should be used. However, if the weights cannot be reliably estimated they may cause weighting may cause larger uncertainty than the use of the unweighted linear regression If the systematic error (or difference between the two sets of results) can be modeled, it can be removed from the results, i.e., the two methods are intercalibrated. A4 B4 c c drdr d

64. Examples Example 1: AAS-determination of Pb from geological samples in two laboratories were compared. Lab 1 used nitric acid extraction and Lab 2 (reference) aqua regia extraction. Sample 15 is left out of calculation as an outlier.

65. 1.522.533.544.5 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 lg x ref, lg x dr1 dr2dr1 dr2 The relative differences plotted vs. log of concentrations are linear.

66. Results from linear regression For inter-calibration of the two methods the following equations can be used: Conclusion: If the systematic error can be modeled it can also be corrected mathematically.

67. 050100150200250 -20 0 20 40 60 80 100 STANDARD METHOD, mg/l dr (%) EXAMPLE 1. Systematic error in BOD determination Relative differences of the company method from the reference method in BOD: Value of this figure in 1984 was 20 000 000 FIM (FIM = 5.95 €) Published in: Pia Bruce, Pentti Minkkinen and Marja-Liisa Riekkola, Practical Method Validation: Validation Sufficient for an Analysis Method, Mikrochimica Acta 128 (1998) 93-106. 1 1.Acceptable BOD concen- tration in discharge, when the new method was adopted 2 2.Acceptable BOD concen- tration in discharge in 1984

68. Story behind Example 1 The company changed its BOD measurement method into an easier procedure than the one that was used in the environmental legislation as reference procedure. The method was tested analysing samples by both method and the new procedure produced compatible results with the reference procedure. At this time the environmental discharge permit was not exceeded at the concentration level of 130 mg/L. With increased production and tightening environmental legislation the maximum allowed concentration that did not exceed the permit was about 90 mg/L. When data that was available (results from inter-laboratory comparison and analysing synthetic samples) was plotted as in Fig. 1 – relative deviations of the company results from known concentrations or from reference procedure, it was obvious that the company procedure had a concentration dependent systematic error; appr. 30 % at the concentration level that the company was operating. When these results were presented to the management 20 million FIM emergency investment plan of upgrading the treatment facility was cancelled, because it had been based on wrong analytical data. The reference procedure was adopted for BOD determinations. Seven years later a modern wastewater treatment plant was built – this time based on correct analytical data.

69. Example 3 Pulp mill was feeding a paper mill through a pipeline pumping the pulp at about 2 % consistency. The total mass of pulp was estimated based on the measurement of a process analyzer installed in the pipe line immediately after the slurry pump at the pulp factory. Material balance calculations showed that something was wrong with the measurement system. Auditing the system showed that the results could be 10 % too high. Both the calibration of method of the analyzer and the sampling system needed to be improved. VALUE OF THE MEASUREMENT ERROR UP TO: 0.1·(100 000 ton/a)·700 $/a = 7 000 000 $

70. Conclusions  Analytical uncertainty has to be monitored continuously.  Sampling and sample preparation usually are the largest error sources and, consequently, sampling has to be included in quality control of analytical measurement.  Good quality control program requires resources, but can also result in considerable savings in the long run.

71. References 1. Gy, P.M., Sampling of Particulate Materials, Theory and Practice, Elsevier, Amsterdam, 1982. 2. Gy, P.M. Sampling of Heterogeneous and Dynamic Material Systems, Elsevier, Amsterdam, 1992. 3. Pitard, F.F., Pierre Gy's Sampling Theory and Sampling Practice, Heterogeneity, sampling correctness, and statistical process control, CRC Press, Boca Raton, 1993. 4. Gy, P. M., Sampling for Analytical Purposes, John Wiley & Sons Ltd, Chichester, 1998. 5.Gy, P. M., Sampling of Discrete Materials, Chemometrics and Intelligent Laboratory Systems, 74 (2004) 7-47. 6. Statistical Aspects of Sampling from bulk materials - Part 1: General principles, ISO 11648-1, Part 2: Sampling of particulate materials, ISO 11648-2, 2001. 7. Opas näytteenoton teknisten vaatimusten täyttämiseksi akkreditointia varten, FINAS S51/2000, Mittatekniikan keskus, Helsinki 2000. 8. Minkkinen, P., SAMPEX - A computer program for solving sampling problems, Chemometrics and Intelligent Laboratory Systems, 7 (1989) 189-194. 9. Minkkinen, P., Practical Applications of Sampling Theory, Chemometrics and Intelligent Laboratory Systems, 74 (2004) 85-94. 10.Eurachem/EUROLAB/CITAC/Nordtest Guide: Estimation of measurement uncertainty arising from sampling, 2006.

72. THANK YOU