Laboratory QA/QC An Overview

Definitions (1) Quality Assurance: QA is defined as the overall program that ensures the final results reported by the laboratory are correct. QA is a broad plan for maintaining quality in all aspects of a program. QA establishes the need for quality control.

Definitions (2) Quality Control: The measures that must be included during each laboratory procedure to verify that the test is working properly. QC refers to routine technical activities with the purpose to control error. QC can be considered as the “HOW” of the QA process. QC is applicable to field, lab and office procedures (administration).

Definitions (3) Quality Assessment - quality assessment (also known as proficiency testing) is a means to determine the quality of the results generated by the laboratory. Quality assessment is a challenge to the effectiveness of the QA and QC programs. Quality Assessment may be external or internal.

Why QC? “QC aims at simply ensuring that the results generated by the test are correct. However, QA is concerned with much more. It checks whether the right test is carried out on the right sample, and that the right result and right interpretation is delivered to the right person at the right time”

Why QA? We need QA to: Understand data reliability; Quantify areas of analytical uncertainty; and Standardize measurement to allow for repeatable and comparable data across time and place.

QA vs. QC broad program plan establishes the need for QC Quality Assurance (QA) broad program plan establishes the need for QC Quality Controls (QC) individual checks and balances the “How to” of QA

Where QC is applicable? Quality control is applicable in all aspects of a soil, plant and water sampling project including: Field data collection and sampling Laboratory analysis and processing Data evaluation and assessment Reporting and project documentation QC provides steps to ensure lab data will meet defined standards of quality with a standard level of confidence

QC in the Field In most cases field QC (soil, water and plant sampling) is out of laboratory control; QC is particularly critical in field data collection; Often the most costly aspect of any project and the most limiting factor is field sampling; Data is never reproducible under the exact same condition or setting; Therefore, field sampling QA is also needed to assure that best possible (most reliable) set of data is obtained.

Choice of Sampling Unit - What Does a Sample Represent? 7.5 cm core Irrigation project Representing 10 ha

QC in the laboratory Laboratory data analysis, data measurement, and data acquisition: Chain of custody forms Equipment calibration Storage practices Analytical methods Holding times Limit of detection (LODs), previously known as MDLs. www.odc.gov/noeh/dls Once field data is collected, it must be evaluated with methods and standards that meet the quality requirements of your environmental investigation. Every aspect of analysis needs to be evaluated; form assuring that the samples being analyzed belong to your project (chain of custody), to preventing potential contamination from other sources (storage practices), to assuring the analytical result will be relevant (exp: the equipment is properly calibrated, and testing is conducted within holding times and using proper methods and levels of detection (minimum detection limits - MDLs) sufficient to quantify what it is you are assessing). Again, the goal is to maintain quality in all aspects of the analytical program.

Variables affecting results quality Educational background and training of personnel; Condition of the samples; Controls used in the test runs; Reagents quality; Maintenance status of equipment; Interpretation of the results; Recording of results; and Reporting of results.

Errors in measurement True value: This is an ideal concept which practically cannot be achieved. Accepted true value: The value approximating the true value, the difference between the two values should be negligible (not statistically significant). Error: The discrepancy between the result of a measurement and the true (or accepted true value).

Sources of error Input data required: Such as standards used, calibration values, and values of physical constants; Inherent characteristics of the quantity being measured; Instruments used: Accuracy, repeatability; Observer unreliability: Reading errors, blunders, equipment selection, analysis and computation errors;

Sources of error (cont.) Environment: Any external influences affecting the measurement; and Theory assumed: Validity of mathematical methods and approximations.

Random Error An error that varies in an unpredictable manner, in magnitude and sign, when a large number of measurements of the same quantity are made under effectively identical conditions. Random errors create a characteristic spread of results for any test method and cannot be accounted for by applying corrections. Random errors are difficult to eliminate, but repetition reduces the influences of random errors.

Random Error (cont.) Examples of random errors include: errors in pipetting; changes in incubation period; or the time used for extraction/centrifuging. Random errors can be minimized by training, supervision and adherence to standard operating procedures (SOPs).

Random Errors

Systematic Error An error that, in the course of a number of measurements of the same value of a given quantity, remains constant when measurements are made under the same conditions, or varies according to a definite law when conditions change. Systematic errors create a characteristic bias in the test results and can be accounted for by applying a correction. Systematic errors may be induced by factors such as variations in incubation temperature, change in the reagent batch or modifications in testing methodology.

Systematic Errors

QC: Internal vs. External Measures Internal Quality Control: “Controllable” by those responsible for performing the laboratory analysis. External Quality Control: A “set of measures” established for and conducted by people outside the analytical laboratory (lab auditors, regional or national laboratories, accreditation process, etc).

Quality control (QC): Internal Internal Quality Control: Equipment calibration Proper training and certification of practitioners Proper sampling techniques Proper data documentation Photos are from the following sources: 1.) http://www.ars.usda.gov/is/np/fnrb/fnrb1001.htm – asbestos containment chamber 2.) http://il.water.usgs.gov/adcp/photos/class_apr99.html - recording data in the field QC information has been adopted from U.S. Environmental Protection Agency Volunteer Monitor’s Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm Some examples of internal quality control measures implemented by the group responsible for the project (including but not limited to the collection and evaluation of samples and data) would include: proper equipment calibration, training and certification of field technicians and other staff, proper documentation of all applicable information (see photo 2 – field logging of collected data), prevention of cross contamination when collecting samples and data (see photo 1 – asbestos chamber) Can your group list examples of other internal measures of “error” control?

Internal Quality Control Samples IQC samples comprises either In-house prepared aliquot of known values, or International standards with values within significant ranges for the element to be measured.

Quality control (QC): External External quality control: Performance audits Split sample analysis Replicate (duplicate) sample analysis External QC - “set of measures” established for those “outside of the program” Examples of external QC controls include: performance audits conducted by outside personnel, usually agency representatives or hired consultants split sample analysis - where one sample is divided equally and sent to more than one laboratory (or annalist) for analysis, and collection of “replicate” samples (or “duplicate” samples when collecting only 2 samples) - two or more samples taken from the same location, at the same time, using the same method, but independently analyzed; generally by a person(s) from outside the organization conducting the investigation.

QA/QC: Data objective and key concepts Successful data collection and analysis is dependant upon “The PARCC Parameters”: Precision Accuracy Representativeness Completeness Comparability “The PARCC Parameters” include: Precision Accuracy Representativeness Completeness Comparability The PARCC parameters are the WHY of the QAPP and QA/QC. The QAPP is developed in order that the key concepts of a well run and thorough environmental investigation may be met. The key concepts of QA/QC are the “PARCC” Parameters – the WHY of the QA

Key concepts of QA/QC: Precision - Accuracy - degree of agreement there is between repeated measurements of the same characteristic can be biased – meaning there is a consistent error in the results Accuracy - measures how close data results are to a true or expected value – does not allow for bias Photo taken from http://www.d.umn.edu/faculty/ UMD med school – lab worker Key concept definitions have been adopted from U.S. Environmental Protection Agency Volunteer Monitor’s Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm Precision is the degree of agreement among repeated measurements of the same characteristic on the same sample or on separate samples collected as close as possible in time and place. Precision gives an idea of how constant and reproducible field or laboratory data are. However, this is only an indication of how consistent results are under similar conditions, and does NOT necessarily mean that results are a “true” value or accurate. (See target diagram on the following slide) Precision may have a BIAS which is the degree of systematic error present in a process. Meaning, that when bias is present the result may be close together (precise) but will differ from the true value or expected result. (See target diagram on the following slide) Accuracy is a measure of confidence in measurement. The smaller the difference between the measurement of a parameter and it’s true or expected value (the less bias), the more accurate the measurement and the more reproducible its result. (See target diagram on the following slide)

Key concepts of QA/QC: Accuracy accuracy = (average value) – (true value) precision represents repeatability bias represents amount of error low bias and high precision = statistical accuracy Information adopted from U.S. Environmental Protection Agency Volunteer Monitor’s Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm Accuracy is the level of confidence in measurement. The smaller the difference between the measurement of a parameter and it’s true or expected value (the more accurate the measurement) the more reproducible its result. Accuracy is determined by comparing the result of analysis for a sample to the “known” value for that sample – such as a standard or a spike. Accuracy needs to be considered when choosing a method of analysis – What level of accuracy is needed? Grater accuracy allows for greater Repeatability - provides for a reliable measurement when you need to confirm the consistency of an environmental condition or look at change over time. ( e.g., for reliable interpretation of trends over time – the nitrification of a lake or remediation of a pollutant). Repeatability is usually acquired by specifying and sticking to a particular sampling and analytical method (methods discussed in slides 12 and 14). Accuracy is reflective of both bias and precision (see target diagram). Discuss with the students the need for accuracy to be represented by both high precision and low bias. http://www.epa.gov/owow/monitoring/volunteer/qappexec.html

Key concept of QA/QC: Representativeness extent to which measurements actually represent the true environmental condition or population at the time a sample was collected. Representative data should result in repeatable data Photos: 1.) taken from WOW Model 8 (8-4-03) 2.) taken from http://pubs.usgs.gov/fs/fs-0058-99/ - Minnesota Lake Key concept definitions have been adopted from U.S. Environmental Protection Agency Volunteer Monitor’s Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm Representativeness – It stands to reason that the more accurate the data, the more representative it will be of the “true” or actual conditions at the site or area under investigation. As stated above, the more accurate the data, the more repeatable the results. Representativeness refers more to the sample location and selection. Are you really representing the conditions of the site when you choose specific sample or the specific sample location? (Simple Example: sampling for benthic organisms at the surface of a lake). http://pubs.usgs.gov/fs/fs-0058-99  Does this represent this?? 

Key concepts of QA/QC: Comparability the extent to which data can be compared between sample locations or periods of time within a project, or between projects Photos: 1.) taken from http://www.nrcs.usda.gov/programs/cta/ctasummary.html 2.) taken from http://www.nssl.noaa.gov/publicaffairs/backgrounders/backgrounder_ww.html - Winter data Key concept definitions have been adopted from U.S. Environmental Protection Agency Volunteer Monitor’s Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm Comparability - a method that works as well in fresh water may not work as well in saltwater (marine) environments! Different methods of analysis may have different limitations on their use because of the relative effects of different sample matrices. Another example - samples taken at different times of the day or at different seasons may not be comparable because of changes that occur with changing daylight or temperature. By specifying a consistent method you can compare two of more places or two places in time - why is this important?( Hint: e.g., to allow reliable comparisons in “before and after” measurements, or “upstream and downstream” measurements around an effluent at a point source) Discuss with the students whether on not data collected under these varying climate conditions may or may not be comparable? What sort of sampling events may be affected by these varying conditions. Example: The collection of water samples for volatile organic compounds – how would hot weather conditions effect the samples and their volatile nature. How about in turn cold weather? What if the samples froze? What if collected samples froze and broke the bottles in which they were stored? Discuss how this relates back to the concept of completeness. Discuss how it relates back to the need for QCs to prevent these problems from occurring.  Will similar data from these sites be Comparable ?? 

Review: QA vs. QC Quality Assurance (QA) broad program plan establishes the need for QC Quality Controls (QC) standardized tests and methods the “HOW” of QA Use this slide as a opportunity to briefly review the meaning of QA and QC – What are the key differences and similarities between the two? Quality Assurance - a broad plan for maintaining quality in all aspects of a program, also refers to the overall management system and includes: Organization, Planning, Data collection, Documentation, Evaluation, and reporting activities. All designed to address the success of the environmental assessment. Quality Control refers to the routine technical activities and established check and balances, the essential purpose of which is to control error. QC employs standardized tests and methods used in the field, laboratory, and office to generate quality, consistent, and repeatable data.

Internal QC program for soil and water sample testing An internal quality control program depends on the use of internal quality control (IQC) samples, and using statistical analysis methods for interpretation.

Shewhart Control Charts A Shewhart Control Chart depend on the use of IQC samples and is developed in the following manner: Put up the IQC specimen for at least 20 or more sample runs and record down the readings; Calculate the mean (x) and standard deviations (Sd); Make a plot with the sample run on the x-axis, and concentration readings on the y axis.

Shewhart Control Charts (cont.) Draw the following lines across the y-axis: mean, -3, -2, -1, 1, 2, and 3 Sd; Plot concentration reading obtained for the IQC specimen for subsequent sample runs. Major events such as changes in the reagent batch and/or instruments used should also be recorded on the chart.

What is Shewhart Control Chart? A Shewhart control chart consists of: Points representing a statistic (e.g., a mean, range, or proportion) of measurements of a quality characteristic in samples analyzed at different times [the data]; The mean of this statistic using all the samples is calculated (e.g., the mean of the means, mean of the ranges, or mean of the proportions);

What is Shewhart Control Chart? (cont.) A center line is drawn at the value of the mean of the statistic; The standard error (e.g., standard deviation) of the statistic is also calculated using all the samples; and Upper and lower control limits (sometimes called "natural process limits"), indicating the threshold at which the process output is considered statistically 'unlikely' are drawn typically at 3 Sd from the center line.

Westgard rules The formulation of Westgard rules were based on statistical methods. Westgard rules are commonly used to analyze data in Shewhart Control charts. Westgard rules are used to define specific performance limits for a particular analysis and can be use to detect both random and systematic errors.

Westgard rules There are six commonly used Westgard rules of which three are warning rules and the other three are mandatory rules. The violation of warning rules should trigger a review of test procedures, reagent performance and equipment calibration. The violation of mandatory rules should result in the rejection of the obtained results.

Shewhart Chart Sample reading Sample run Target value +3 sd +2 sd

Warning rules Warning 12SD : It is violated if the IQC value exceeds the mean by 2SD. It is an event likely to occur normally in less than 5% of cases. Warning 22SD : It detects systematic errors and is violated when two consecutive IQC values exceed the mean on the same side of the mean by 2SD. Warning 41SD : It is violated if four consecutive IQC values exceed the same limit (mean  1SD) and this may indicate the need to perform instrument maintenance or reagent calibration.

Mandatory rules Mandatory 13SD : It is violated when the IQC value exceeds the mean by 3SD. The test is regarded as out of control. Mandatory R4SD : It is only applied when the IQC is tested in duplicate. This rule is violated when the difference in Sd between the duplicates exceeds 4Sd. Mandatory 10x : This rule is violated when the last 10 consecutive IQC values are on the same side of the mean or target value.

Westgard Rules: 1 3SD Sample reading Assay Run Target value +3 sd

Westgard Rules: 10X Sample reading Assay Run Target value +3 sd +2 sd

Follow-up action in the event of a violation There are three options as to the action to be taken in the event of a violation of a Westgard rule: Accept the test run in its entirety: This usually applies when only a warning rule is violated. Reject the whole test run: This applies only when a mandatory rule is violated. Enlarge the grey zone and thus re-test range for that particular test run: This option can be considered in the event of a violation of either a warning or mandatory rule.