CHOICE OF METHODS AND INSTRUMENTS Part 2

3- Method Evaluation Interference Experiment Interference studies will determine if specific compounds affect the accurate determination of analyte concentrations. Common interferences include hemolysis and turbidity, which can obscure the absorbance of the measured analyte. Interferents, however, either can react with the analytic reagent or may alter the reaction between the analyte and the analytic reagents.

3- Method Evaluation Interference Experiment The interference experiment is performed to estimate the systematic error caused by other materials that may be present in the specimen being analyzed We describe these errors as constant systematic errors because a given concentration of interfering material will generally cause a constant amount of error regardless of the concentration of the analyte being tested in the specimen As the concentration of interfering material changes, however, the size of the error is expected to change

3- Method Evaluation Interference Experiment The volume of the interferer should be small relative to the original test sample to minimize the dilution of the patient specimen However the amount of dilution is not as important as maintaining the exact same dilutions for the pair of test samples Using the test method in question, a sample is analyzed in triplicate to determine a value A specific amount and concentration of a substance thought or known to be an interfering substance is added to this sample and the sample is then reanalyzed

3- Method Evaluation Interference Experiment The difference between the result of the sample with and without the substance is calculated The sample is repeated with several different concentrations of the interfering substance The acceptability of this experiment, according to Westagrd, is that the difference between the two values, one with and the other without interfering substance, should be less than the allowable analytical error at that concentration

3- Method Evaluation Interference Experiment Interferences to be tested The substances to be tested are selected from: the manufacturer's performance claims, literature reports, summary articles on interfering materials, and data tabulations or databases. It is also good practice to test common interferences such as bilirubin, hemolysis, lipemia, and the preservatives and anticoagulants used in specimen collection. Less common substances such as Drugs

3- Method Evaluation Interference Experiment Interferences are classified as endogenous or exogenous Endogenous interference originates from substances found naturally in the patient sample. Haemolysis (haemoglobin and other substances), bilirubin, lipids, proteins, antibodies (autoantibodies, heterophile antibodies) Exogenous interference results from substances not naturally found in the patient’s specimen, including drugs (parent drug, metabolites, and additives), poisons, herbal products, IV fluids, substances used as therapy (e.g. antibodies, digibind). It may also arise from collection tube components Digibind is an antidote for digoxin toxicity. Heterophile antibodies are antibodies induced by external antigens (heterophile antigens).

3- Method Evaluation Interference Experiment Examples of Interferences to be tested Bilirubin can be tested by addition of a standard bilirubin solution Hemolysis is often tested by removing one aliquot of a sample, then mechanically hemolyzing or freezing and thawing the specimen before removing a second aliquot. Lipemia can be tested by addition of a commercial fat emulsion, such as Liposyn (Abbott Laboratories) or lntralipid (Cutter Laboratories) Additives to specimen collection tubes can be conveniently studied by drawing a whole blood specimen, then dispensing aliquots into a series of tubes containing the different additives

3- Method Evaluation Interference Experiment As example, determination of glucose will be used Creatinine is an interfering substance in this method. By adding creatinine to a sample in measured amounts, the amount of interference can be determined Prepare the sample as follows: Each sample is prepared and run in triplicate and the results are averaged. Sample 1 0.9 ml serum 0.1 ml deionized water (baseline) Sample 2 0.1 ml of 50 mg/dl creatinine STD Sample 3 0.1 ml of 100 mg/dl creatinine STD

3- Method Evaluation Interference Experiment Results:   Amount of glucose measured Amount of Creatinine added Interference Sample 1 100 mg/dl 0 mg/dl Sample 2 104 mg/dl 5 mg/dl + 4 mg/dl Sample 3 111 mg/dl 10 mg/dl +11 mg/dl

3- Method Evaluation Interference Experiment Calculations: Amount of creatinine added (CV=C1V1) Amount of Interference Interference = Conc. Measured (test) — Conc. Measured (baseline) Interference = 104 mg/dl — 100 mg/dl = 4 mg/dl Conc. Added = STD. Conc. X ml STD. Added ml of serum+ ml of STD Conc. Added = 50 mg/dl X 0.1 ml = 5 mg/dl 0.1 ml + 0.9 ml

3- Method Evaluation Interference Experiment Interpretation: The experiment shows that increasing amounts of creatinine will result in an increasing positive interference The acceptable level of interference and accuracy will vary depending on the analyte and the concentration The judgment on acceptability is made by comparing the observed systematic error with the amount of error that is allowable for the test. For example, a glucose test is supposed to be correct to within 10% according to the CLIA proficiency testing criteria for acceptable performance. The upper end of the reference range (110 mg/dL)

Experiments and type of error which can be detected

3- Method Evaluation Comparison Of Methods A comparison of methods study is done to estimate systematic error (inaccuracy or bias) within the new method. If bias is present, and if the appropriate statistical calculations are done, systematic error will be identified as constant, proportional, or a combination. A comparison of methods experiment examines patient samples by the method being evaluated (Test Method) with a reference method (Comparative Method).

3- Method Evaluation Comparison Of Methods Ideally, the test method is compared with a standardized reference method (gold standard), a method with acceptable accuracy. Many times reference methods are laborious and time consuming, as is the case with the ultracentrifugation methods of determining cholesterol. Because most laboratories are not staffed to perform reference methods, most test methods are compared with those routinely used.

3- Method Evaluation Comparison Of Methods When using a known reference method, all of the observed differences between the methods can be attributed to the new method. When the comparative method is the one that is being used and the bias and analytical error is known, then part of the observed analytical error noted between the two methods can be attributed to the comparative method, with the remaining error belonging to the test method. The difference between the two methods at medical decision levels should be less than the allowable analytical error for that concentration of the analyte.

3- Method Evaluation Comparison Of Methods This experiment involves simultaneously analyzing split samples on both the test and comparative method Only patient samples should be used for evaluation between methods Lyophilized, aqueous, or ethylene glycol-based control samples have slightly different physical properties than fresh human serum or plasma and may react differently in different systems Westgard recommends that each sample be run in duplicate, at different analytical runs This is done in order to check on the validity of the experimental observations and to detect random errors, such as mixing up of samples and short sampling

3- Method Evaluation Comparison Of Methods The replicates should be analyzed the same day on the same instrument and within as short a time as possible The difference between the duplicate measurements should be less than or equal to the determined between-run precision If the differences are greater than this, suspect an error Analyze a minimum of 40 samples, 5 samples a day for 8 days Select a variety of concentrations so that the entire linear or working range of the new method will be represented

3- Method Evaluation Comparison Of Methods Reduce opportunities for bias and mistakes to occur: Limit the number of technologists participating in the experiment. Try to run samples on both instruments at the same time to limit the effects of sample deterioration. Use acceptable techniques in preparing reagents and samples for analysis.

3- Method Evaluation Comparison Of Methods There are several ways of approach in evaluating the comparison between two methods: Plot the results on a scattergram Linear regression analysis The paired t-Test The F-Test

3- Method Evaluation Comparison Of Methods Plot the results on a scattergram One of the easiest and most visually studied is to plot the results on a scattergram, draw a best straight line through the points, and determine the slope and intercept of the line The slope and the intercept offer a fair evaluation of the agreement between the methods Data should be plotted on the scattergram on a daily basis and inspected for outliers so that original samples can be reanalyzed as needed If the technologist waits until all of the samples are analyzed there will be no opportunity to investigate outliers or to reanalyze samples

3- Method Evaluation Comparison Of Methods Plot the test method on the y-axis and the reference or comparative method on the x-axis. A best fit straight line can be drawn through the plotted points A visual inspection of the line and the points around it can give a preliminary assessment of the agreement between the methods Comparative method Test method

3- Method Evaluation Comparison Of Methods Linear regression analysis: Linear regression analysis statistics are useful in calculating systematic error. It calculates the best straight line through the data points, calculates the slope and intercept of the line with the y-axis, and the standard deviation or error of the points about the regression line Calculation of the slope and intercept will show how well the methods agree If the test method matches the comparative method’s precision and accuracy, the slope of the best straight line through the points should have a value of 1.000 and a zero y-axis intercept

3- Method Evaluation Comparison Of Methods A plot of the test-method data (y-axis) versus the comparative method (x-axis) helps to visualize the data generated in a Comparison of methods test. If the two methods correlate perfectly, the data pairs plotted as concentrations values from the reference method (x) versus the evaluation method (y) will produce: a straight line (Y = mX + b), [m is the slope and b is the y-intercept] with a slope of 1.0, a y-intercept of 0, and a correlation coefficient (r) of 1 It indicates the extent of linear relationship between the methods Method accuracy should not be based on r

3- Method Evaluation Comparison Of Methods Generate a “linear best fit line” Y= mX + b m = slope (indicates a proportional error) b = intercept (indicates constant error) Evaluate linear regression line: Evaluate slope Slope = 0.900 = -10% proportional error Slope = 1.100 = +10% proportional error Intercept should be close to zero (indicating constant bias)

3- Method Evaluation Comparison Of Methods Comparative method Test method Perfect correlation (Hypothetical) Slope (b) = 1 Y intercept (a) = 0 Correlation coefficent (r) = 1

3- Method Evaluation Comparison Of Methods The systematic error (SE) at a given medical decision concentration (Xc) is then determined by calculating the corresponding Y-value (Yc) from the regression line, then taking the difference between Yc and Xc, as follows: Yc = mXc + b SE = Yc - Xc For example, given a cholesterol comparison study where the regression line is Y = 1.03X + 2, i.e., the y-intercept is 2.0 mg/dL and the slope is 1.03, the Y value corresponding to a critical decision level of 200 would be 208 (Y = 2.0 + 1.03*200), which means there is a systematic error of 8 mg/dL (208 – 200) at a critical decision level of 200 mg/dL.

3- Method Evaluation Comparison Of Methods Linear regression analysis has limitations. The slope, intercept, and standard deviation of the regression line can be influenced by: Nonlinearity of either method; Outliers values, and a narrow range of samples These limiting factors can be discovered by: Careful analysis of the plotted data and a follow-up investigation. Careful choice of samples to cover the entire working range of the test method is important The elimination of outliers caused by mistakes

3- Method Evaluation Comparison Of Methods Linear regression has one other defect. It assumes that the comparative method is truth, in fact, the analytical error observed between the two methods may be largely due to error in the comparative method Therefore it is extremely important that the accuracy and precision of the comparative method be well known