1. Module 6: QC Basic Rules and Charts
Analysis, Charts
and Interpretation of Rules
Quality Control: Samples, Charts and Rules

2. Learning Objectives
Upon completion of this module you will be able to:
Define: Control materials, Calibration materials, Levey-Jennings Chart, Westgard Single Rule and Multi-Rule Criteria, Systemic Error, & Random Error
Describe a quality control sample in terms of assayed versus unassayed and their uses

3. Learning Objectives
Upon completion of this module you will be able to:
Discuss the following Westgard rules:
13s
12s
Apply the Westgard rules listed above to decide whether an analytic run is accepted or rejected.
List which Westgard rules detect systematic versus random error.
22s 6x 7t
Quality Control: Samples, Charts and Rules

4. Definitions
Control materials: Protein-based materials made or manufactured with a composition similar to patient samples that have or have not been analyzed for concentration so they have expected or no expected values.
The International Federation of Clinical Chemistry defines a control solution or control material as a "specimen or solution which is analysed solely for quality control purposes, not for calibration" .
Quality Control: Samples, Charts and Rules

5. Definitions Certified reference material (CRM)
“A reference material that has one or more values certified by a technically valid procedure and is accompanied by, or is traceable to, a certificate or other document that is issued by a certifying body.” [CLSI] NBS (National Bureau of Standards)
Calibrator: purified solution used to adjust electrical output of an instrument and should trace back to a CRM.
The International Federation of Clinical Chemistry defines a control solution or control material as a "specimen or solution which is analysed solely for quality control purposes, not for calibration" .
Quality Control: Samples, Charts and Rules

6. Definitions
Levey-Jennings Chart: A visual presentation of daily quality control values plotted on a chart using mean, standard deviation, and +/- 3 Std Dev range criteria
Westgard Single Rule and Multi-Rule Criteria: A set of rules used to evaluate quality control values as acceptable or not
The International Federation of Clinical Chemistry states that calibrators are purified solutions used to adjust the output of an instrument to a particular concentration of an analyte.
Quality Control: Samples, Charts and Rules

7. Definitions
Systemic Errors: Errors in the test system usually caused by a malfunction that affects all tests
Random Errors: Errors that statistically occur unpredictably and do not indicate system malfunction but do indicate an individual test or component malfunction

8. Need for Quality Control Samples
Patient results are “unknowns”. Analysis of the patient sample produces a number.
But…is the result valid?
Patient may have previous result. But…
Change from previous may be expected
No change from previous result may be expected
The product of a testing process is a numerical result. Unlike a physical product that can be inspected to assess whether it looks good or bad, you can't look at a test result and tell whether it's valid. You could look to see if that patient had the same test measured recently but you don’t know whether to expect it to change because the patient has improved or worsened or if you expect it to stay the same. Quality control samples are like patient samples only they have a known amount of analyte. QC samples (knowns) can be analysed along with patient samples (unknowns) to test the validity of all results in the run.
Quality Control: Samples, Charts and Rules

9. Need for Quality Control Samples
Quality control samples are “knowns.” They have been made or manufactured with a composition similar to patient samples and have been analyzed for concentration before they are put into use so they have expected results.
Therefore, if Quality Control samples are analyzed and expected results are obtained, we can assume our system is operating correctly. or
If Quality Control samples are analyzed and do not give expected results, what should be done?

10. Quality Control Samples Are
Preserved to maintain accurate and precise results
Assayed –provided with mean and s for analytes
Unassayed – you must determine mean and s
The purpose of a statistical quality control procedure is to monitor the analytical quality of the measurement during stable operation, detect changes from the stable operation, and eliminate reporting of results with medically important errors. Control samples should be similar to patient specimens. They can be purchased from manufacturers who make them from donated patient samples that have been preserved. It is important to treat the control materials carefully. Attributes of controls are the stability, low vial to vial variability, assayed versus unassayed, and how to reconstitute or treat the control material.
Quality Control: Samples, Charts and Rules

11. Why is it advisable to run controls at different levels?
Interactive Opportunity
Why is it advisable to run controls at different levels?
Gather input from participants.
One acceptable response is that running a control that has a result falling in the reference range and at least one control that falls at the medical decision level is value to test the analytical phase.
Quality Control: Samples, Charts and Rules

12. Quality Control Sample Use
Analyzed solely for quality control purposes
Not for calibration
Should be used for qualitative and quantitative testing as known results to check if method to measure patient results is reliable.
The International Federation of Clinical Chemistry defines a control solution or control material as a "specimen or solution which is analysed solely for quality control purposes, not for calibration" . Even with assayed controls you should assay the selected control materials under routine operating conditions to characterize the expected measurement variation and establish your own expected values. This usually involves obtaining at least 20 values and calculating the mean and standard deviation. There are a number of pitfalls from using bottle values or other estimates of the means, standard deviations, and control limits without verifying the results first.
Quality Control: Samples, Charts and Rules

13. Interactive Opportunity
Is it sufficient to use only one calibrator to set the output of an analyzer?
Gather input from participants.
One acceptable answer is that calibration should initially be with standards that test the lowest, mid and highest level of linearity. Validation of the curve using one calibrator is sufficient followed by analysis of quality control materials.
Quality Control: Samples, Charts and Rules

14. Qualitative Versus Quantitative Results
What are some examples of qualitative tests in the laboratory?
What quality control samples would you use to verify the test is working so that patient results may be reported?
Take answers from participants and put on flipchart. Examples: HIV with + and neg controls, RPR or VDRL with reactive and NR controls, ABO with A1 cells (positive for A), B cells (positive for B) O cells or antibody screen cells (negative for A and B), Gram stain (E coli for GNB and Staph for GPC), many others may apply. Most urine testing and hematology testing is quantitative and should be reminded of need for analyzing known control samples or blood films even for manual microscopy to check technician results.

15. QC Results and Expectations for Qualitative Tests
Quality Controls will be a known positive and known negative material
The results will be recorded in a log book with the date, techs initials, and either ‘positive’ or ‘negative’ written in by the tech
Reason for running controls with qualitative tests is to check that the reagent(s) are working as they are supposed to and give accurate results
Depending on the test, the (+) and (-) controls will be run once a day (usually every morning) or each time a patient specimen is analyzed. When running a urine dipstick test, the controls are usually run only in the morning and recorded; for other tests, HIV, RPR, etc. the (+) and (-) controls must be run along with the patient to ensure reagents have been added correctly. (with the HIV kits, the controls are build in to the test strip)
If the (=) and (-) controls do not give the expected result you must discard the patient test and repeat both the controls and the patient.
Quality Control: Samples, Charts and Rules

16. QC Results and Expectations for Quantitative Tests
Gaussian distribution
In the laboratory, control charts are used to make it simple to compare today's observed value with what is expected based on past history. Assuming a Gaussian or normal bell-shaped distribution, it would be expected that about 68% of the points fall within 1 SD of the mean, 95% within 2 SD of the mean and 99.7% within 3 SD of the mean. A control chart also has the mean and standard deviations indicated but they are shown on the y axis (as if you turned this Gassian distribution on its side.)
Quality Control: Samples, Charts and Rules

17. What does the “Internal” Quality Control on a test cartridge indicate?
Interactive Opportunity
What does the “Internal” Quality Control on a test cartridge indicate?
In some test systems it indicates only that sufficient sample has reached all necessary parts of the test cartridge. If you add sufficient water you would get the indication for + and -. Some qualitative immunoassay methods such as rapid HIV tests have internal quality control to test for the ability of the test strip to detect a positive result.
Quality Control: Samples, Charts and Rules

18. QC Sample Analysis as an Alarm System
QC sample result should:
Indicate when the test system is not working properly
Not indicate a false alarm
It should be 100% reliable
Quality control is run to see if you can get the right answer for a known sample. The right answer is actually a range of values that are calculated from the mean and standard deviation of past results. That mean and control limits can be shown on a control chart to make it simple to plot new control measurements and see how they compare with the expected range of values. From the technologist’s standpoint, the objectives of the control procedure are simply to "alert me when the method has a problem" and "don't alert me when the method is working okay." These are "true alarm" and "false alarm" situations. Technologists want to know about real problems, but can't afford to waste time when the method's working okay.
Quality Control: Samples, Charts and Rules

19. Common QC Sample Rules Single Rule:
Accept the analytical run if both quality control results fall within mean +/- 2 standard deviations range.
Reject the analytical run and troubleshoot the problem if one or both quality control results are outside +/- 2 standard deviations from the mean.
Quality Control: Samples, Charts and Rules

20. QC Sample Result Outside of Expected Range
QC value < +2 SD or > -2 SD from the mean
Expected most of the time
With 95% confidence. This means 95% of acceptable results are in this range. 5% could still be acceptable i.e. false alarm
QC value > + 3 SD or < - 3 SD from the mean
Usually indicates a problem and should be followed up on
It would be very unexpected (0.3% chance) to observe a control value greater than 3 SD from the mean and such an observation usually indicates there is a problem with the method. It is somewhat unexpected to observe a control value greater than 2 SD from the mean, but this will happen at least 5% of the time when analysing 1 control per run, so it may indicate a real problem or it may be a false alarm. It is very common (32% chance) to see individual values beyond 1 SD from the mean, therefore this control limit is of no value for making a judgment about method performance based on a single control value.
Quality Control: Samples, Charts and Rules

21. Historical Use of QC Sample Results
Shewart developed statistical tools to monitor quality
Levey and Jennings – quality standards for the laboratory
Westgard - established rules for monitoring quality control
Walter A. Shewhart was a statistician at Bell Telephone Laboratories who developed the scientific basis for statistical process control. He set up limits within which the results must lie and found that deviations in the results of a routine process outside such limits indicate that the routine has broken down. Levey and Jennings introduced statistical control methods in clinical laboratories, calculating the average and range (maximum difference), then plotting the average and the range on two different control charts. Levey and Jennings also proposed making duplicate measurements on a patient specimen. Others suggested using a stable reference sample and plotting answers directly on a control chart, the Levey-Jennings chart.
Quality Control: Samples, Charts and Rules

22. Levey-Jennings Chart
A Control chart is a graphical method for displaying control results and evaluating whether a measurement procedure is in-control or out-of-control. Control results are plotted versus time or sequential run number; lines are generally drawn from point to point to accent any trends, systematic shifts, and random excursions.
Control limits are lines drawn on a control chart to provide graphical criteria for assessing whether a measurement procedure is in-control or out-of-control.
Quality Control: Samples, Charts and Rules

23. Levey-Jennings Chart

24. Quality Control Result Limits
Mean is the central line.
One standard deviation above = + 1 s
One standard dev below = - 1s
2 standard dev above = + 2s
2 standard dev below = -2s
3 standard dev above = + 3s
3 standard dev below = -3s
These control limits are usually calculated from the mean and standard deviation (SD, or s) determined for a given control material. Typically the interpretation is based on a specified number of results or points exceeding a certain control limit when in-control patient test results are reported. When out-of-control, the run is rejected and no test results can be reported.
Quality Control: Samples, Charts and Rules

25. Calculating Confidence Limits on the Chart
For example, Control 1 has a mean of 200 and a standard deviation of 4 mg/dL.
What is the 95% confidence limit or mean + 2 s limit?
The upper control limit, mean + 2s would be:
*4 = 208 mg/dL
The mean – 2s, lower control limit would be:
*4 = 192 mg/dL
95% confidence limit is mg/dL
This should be a practice problem for participants to set 95% confidence limits. Click to get answers after participants have offered up their answers.
Quality Control: Samples, Charts and Rules

26. Quality Control Sample Rules
Decision Criteria
In control versus out of control
Control limits
Mean + 2s
95% limits
5% false rejection
If 1 result exceeds 2s but not 3s and no other shifts or trends, it is usually a warning but not a reason to reject the results.
Control rule means decision criteria for judging whether an analytical run is in-control or out-of-control. It is defined by the number of control measurements, and control limits, often specified as the mean plus or minus a multiple of the standard deviation (s) or sometimes by a specified probability for false rejection. Mean +/- 2s give 95% change of correct decision but 5% false rejection.
Quality Control: Samples, Charts and Rules

27. QC Sample Measured with each Analytical Run
How long is a run?
Manual
Automated method
Run, analytical run, or run length refer to the interval, which could be a period of time or group of samples, for which a decision on control status is to be made. Each batch is a run for a manual procedure. 2 controls should be run with each batch. 24 hours is usually the longest run time for automated chemistry methods; 2 controls should be run at least once every 24 hr. 8 hours is the longest run time for automated hematology methods; 3 controls should be run at least once every 8 hr.
Quality Control: Samples, Charts and Rules

28. QC Sample Analysis Procedure
Laboratory Quality Control Policy (Quality Manual)
stating your laboratory will not report results without
appropriate QC.
Quality control Analysis SOP
How many control samples
How often run
How documented
How problems are solved
What Westgard rules are used.
You should prepare written guidelines to define the QC procedure in detail including how many controls to run per analyte, how often they are run, how to document results and problems and how to solve the problems. This written document is important for teaching laboratory techs the QC procedure and establishing a uniform practice. It is also necessary for meeting regulatory requirements.
Quality Control: Samples, Charts and Rules

29. Lab Results Need to Be... Accurate Precise
Quality Control Rules should help to assess accuracy and precision.
After verification with QC Rules, lab results also need to be reported timely.

30. Accuracy Closeness of the measured result to the true value
Your control values may show precision but be inaccurate
Instrument can repeatedly get the same value, but it’s not the right value
This diagram is imprecise, but accurate

31. Precision
Precision is: The reproducibility or closeness of results to each other
The degree of fluctuation on repeated measurements is indicative of the “precision” of the assay.
The closeness of measurements to the true value is indicative of the “accuracy” of the assay.
This is inaccurate, but is precise

32. Interactive Opportunity
How can we check for accuracy and precision?
Can you have precision without accuracy?
Can you have accuracy without precision?
STRESS to participants that Quality Control:
Should detect random errors (Imprecision)
Should detect systematic errors (Inaccuracy)
Shift (6 results that settle around new mean)
Trend (7 results upward or downward )
Quality Control: Samples, Charts and Rules

33. Possible Causes for Random Errors
Can be due to variations in line voltage, pipetting, dispensers, contamination, volume dispensed, bubbles in lines of reagents, etc*…
A random error can be either positive or negative
The direction and exact magnitude of the random error cannot be exactly predicted
Random Errors affect reproducibility or precision of a test system
*NOTE: These variations should be detected during morning instrument start-up and maintenance
*NOTE: Pipettes with poor accuracy can cause systematic errors also.
*NOTE: When you see a random error on QC results, the technologist should look for the possible causes listed in the slide to begin the troubleshooting before re-running controls.

34. Random Errors and Possible Causes
1 3s Rule Violation:
Usually indicates a Random Error
When reviewing statistical data remember when using +/- 3 SD, there is a 0.3% chance of getting a control value outside the +/- 3 SD range
Before running patient specimens you must troubleshoot AND then rerun the control and the values must be within +/- 2SD
Check that controls are in date and well mixed*
*NOTE: Remind participants to include detailed steps to correctly reconstitute control material in their policy statement and procedure for Section 5 (Equipment) in their Quality Manual
Quality Control: Samples, Charts and Rules

35. Westgard Rules Random error detected by 1 3s
This means that 1 QC value exceeds -3s
Could be below -3s or above + 3s
13s refers to a control rule that is commonly used with a Levey-Jennings chart when the control limits are set as the mean plus 3s and the mean minus 3s. A run is rejected when a single control measurement exceeds the mean plus 3s or the mean minus 3s control limit and it indicates a random error. This indicates the need to stop and look for the cause of the random error, solving the problem before reporting the patient results.
How can you determine it is random error on the first violation? May be a sudden shift or trend?
Quality Control: Samples, Charts and Rules

36. Possible Causes for Random Errors
2 2s Rule Violation:
Usually indicates a Systematic Error
Remember, the first time single control value is > or < 2 SD it serves as a warning and you can run and report out your patient values
BUT, the second sequential time the control value is > or < 2 SD, you must trouble shoot and find the reason for the error BEFORE you can run patient specimens and report out patient values
Check that controls are in date and well mixed
*NOTE: 12s is only a warning while 22s rule violation within 20 days signals the run cannot be accepted and the test must undergo troubleshooting to detect the error BEFORE patient specimens may be analyzed.
Quality Control: Samples, Charts and Rules

37. Westgard Rules Systematic error detected by 2 2s
Could be below -2s or above + 2s

38. Systematic Errors Systematic Errors include bias/shifts, & trends
These errors affect accuracy of the test system.
Usually 6x and 7t Rule Violations
Can be due to calibration lot changes, temperature changes in incubator unit, light source deterioration, electronics, reagent lot changes, etc.
Systematic errors are always in one direction on a L J chart.

39. Bias/Shift: 6x Rule Violation
A systematic difference or systematic error between an observed value and some measure of the truth.
6x Rule violations:
Usually indicate an error with your test system
On day 7, after 6 days of one or more control values above or below the mean*, you must trouble shoot and find/correct error before running patient specimens
*Note the values of the 6 days are within +/- 2sd
FYI:
Generally used to describe the inaccuracy of a method relative to a comparative method in a method comparison experiment.
It also has a specific meaning in the statistical t-test, where bias equals the difference between the mean values of the two methods being compared or the average of all the differences between the paired sample values.

40. Westgard Rules Systematic Error detected with 6x: shift
6 results are on the same side of the mean
If a laboratory uses this rule, on the 6th run in which the result settles around a new mean such as shown here with 6 results below mean and above -2s, a shift has occurred. This indicates the need to stop and look for the cause of the systematic error, solving the problem before reporting the patient results. //// OR, does troubleshooting this have any medical relevance for this particular test? Discuss Platelet counts vs K+ levels
Quality Control: Samples, Charts and Rules

41. Trend: 7t Rule Violation
Systematic error when a series of control values are treated as a time series
7t Rule violations:
Usually indicate an error with your system
On day 8, after 7 days of one or more control values moving either upward or downward, you must trouble shoot and find/correct error before running patient specimens

42. Westgard Rules Systematic error detected with 7t: trend
7 results falling in a pattern, up or down
On the 7th run when an upward or downward trend is discovered even within the 95% limits of mean +/- 2 standard deviations, a systematic error has occurred. This indicates the need to stop and look for the cause of the systematic error, solving the problem before reporting the patient results.
Quality Control: Samples, Charts and Rules

43. Levey Jennings Charts:
Evaluation of Results

44. Westgard Rules: Single or Multi Rule QC Single rule QC
Procedure uses a single criterion or single set of control limits, such as a Levey-Jennings chart with control limits set as either the mean plus or minus 2 standard deviations (2s) or the mean plus or minus 3s. There is more than 1 single rule but each rule is assessed individually.
Multi-rule QC
Uses a combination of decision criteria, or control rules, to decide if an analytical run is in-control or out-of-control
Uses 5 different control rules to judge acceptability of an analytical run
Now for a more technical description. Multirule QC uses a combination of decision criteria, or control rules, to decide whether an analytical run is in-control or out-of-control. The well-known Westgard multirule QC procedure uses 5 different control rules to judge the acceptability of an analytical run. By comparison, a single-rule QC procedure uses a single criterion or single set of control limits, such as a Levey-Jennings chart with control limits set as either the mean plus or minus 2 standard deviations (2s) or the mean plus or minus 3s. "Westgard rules" are generally used with 2 or 4 control measurements per run, which means they are appropriate when two different control materials are measured 1 or 2 times per material, which is the case in many chemistry applications. Some alternative control rules are more suitable when three control materials are analyzed, which is common for applications in hematology, coagulation, and immunoassays.

45. Westgard Rules
Many laboratories follow the Westgard Single-Rule policy.
The rules are assessed individually to check for random and systematic errors.

46. Westgard Multirule System
These rules work best together in a multi-rule system with automated analyzers
Rules evaluated in a certain order
Starts with the warning rule (12s ) that triggers the need to look for other violations:
The first time a result exceeds mean + 2s or mean -2s but is within 3 SD it is a warning to check back for shifts or trends or other rule violations. It should indicate the need to verify this occurs only rarely (5% of the time or no more than once in 20 days) and that a 6 x or 7t aren’t also present. A multirule system means that many rules are used to verify patient results are correct, ruling out both random errors, such as 1 3s and systematic errors such as 2 2s, 6x, 7t or even 4 1s rule violations.
Quality Control: Samples, Charts and Rules

47. Westgard Multirules
If 1 control exceeds 2 s (1 2s rule is violated) this decision tree says this is a warning to look for other rule violations such as the 2nd control exceeding 2 s (2 2s) or R4s to decide whether to accept or reject the run. Rejection means that you should turn to the procedure or instrument operating manual to solve the problem before reporting patient results.
This decision tree helps to decide whether to reject or accept a run. If you answer yes, it means that rule has been violated.
Quality Control: Samples, Charts and Rules

48. Key Points Describe a control in terms of
Assayed (mean and s provided)
Unassayed (mean and s not provided)
Use (validation of run not calibration)
Calculate 95% limits for quality control samples
Detect errors in quality control using 95% and Westgard rules
Quality Control: Samples, Charts and Rules

49. Key Points
Levey-Jennings chart include values and markings on the x (days or time) and y axis (concentration mean and s)
Quality Control: Samples, Charts and Rules

50. Key Points Westgard rules increase ability to detect systemic errors.
Quality Control: Samples, Charts and Rules

51. Key Points Westgard Rules increase ability to detect random errors.
Quality Control: Samples, Charts and Rules

52. Key Points Westgard Rule, 1 2s is provided as a Warning
Quality Control: Samples, Charts and Rules

53. References
Shewhart WA. Economic Control of Quality of Manufactured Product. New York; D. Van Hostrand Company, Inc., 1931.
Levey S, Jennings ER. The use of control charts in the clinical laboratory. Am J Clin Pathol 1950;20:
Henry RJ, Segalove M. The running of standards in clinical chemistry and the use of the control chart. J Clin Pathol 1952;27:
Westgard JO, Groth T, Aronsson T, Falk H, deVerdier C-H. Performance characteristics of rules for internal quality control: probabilities for false rejection and error detection. Clin Chem 1977;23:
Quality Control: Samples, Charts and Rules

54. References
Westgard JO, Barry PL, Hunt MR, Groth T. A multi-rule Shewhart chart for quality control in clinical chemistry. Clin Chem 1981;27:
Westgard JO, Barry PL. Cost-Effective Quality Control: Managing the Quality and Productivity of Analytical Processes. Washington, DC:AACC Press
Quality Control: Samples, Charts and Rules