1. NR 422 Quality Control Jim Graham Spring 2009

2. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information Knowledge Understanding Wisdom Organization Interpretation Verification Selection Testing Comprehension Integration Judgment Environmental Monitoring and Characterization, Aritola, Pepper, and Brusseau

3. Error Data does not match reality (ever) Gross errors Accuracy (bias): distance from truth –| Measurement mean – Truth | Precision: variance within the data –Standard Deviation (stddev) Measurement Limits

4. Accuracy and Precision High Accuracy Low Precision http://en.wikipedia.org/wiki/Accuracy_and_precision Low Accuracy High Precision

5. Bias (Accuracy) Bias = Distance from truth TruthMean Bias

6. Standard Deviation (Precision) Each band represents one standard deviation Source: Wikipedia

7. Other Approaches Confidence Intervals +- Some range (suspect)

8. Sources of Error Measurement Error –Protocol –User –Instrument Processing Errors –Procedure –User –Instrument Data Errors –Age –Metadata/Documentation

9. Protocol Rule #1: Have one! Step by step instructions on how to collect the data –Calibration –Equipment required –Training required –Steps –QAQC See Globe Protocols: –http://www.globe.gov/sda/tg00/aerosol.pdf

10. Protocol Error Is there a protocol? What is being measured? Is it complete: How large? How small? Unexpected circumstances (illness, weather, accidents, equipment failures, changing ecosystems)

11. User Measurement Errors Wrong Datum Data in wrong field/attribute Missing data Gross errors Precision and Accuracy Observer error: expertise and “drift”

12. Instrument Errors Calibration Drift Humans as instruments: –DBH –Weight –Humans are almost always involved! –Fortunately we can be calibrated and have our drift measured

13. Calibration Sample a portion of the study area repeatedly and/or with higher precision –GPS: benchmarks, higher resolution –Measurements: lasers, known distances –Identifications: experts, known samples Use bias and stddev throughout study Also provides an estimate for min/max

14. Flow of error Capture error during data collection Determine error of other datasets –If unavailable, estimate the error Maintain error throughout processing –Error will increase Document final error in reports and metadata

15. Processing Error Error changes with processing The change depends on the operation and the type of error: –Min/Max –Average Error –Standard Error of the Mean –Standard Deviation –Confidence Intervals

16. Combing Bias Add/Subtraction: –Bias (Bias1+Bias2)= T- (Mean1*Num1+Mean2*Num2)/(Num1*Num2) Simplified: (|Bias1|+|Bias2|)/2 Multiply Divide: –Bias (Bias1*Bias2)= T- (Mean1*Mean2) Simplified: |Bias1|*|Bias2| Derived by Jim Graham

17. Combining Standard Deviation Add/Subtract: –StdDev=sqrt(StdDev1^2+StdDev2^2) Multiply/Divide: –StdDev= sqrt((StdDev1/Mean1)^2+(StdDev2/Mean2)^2) http://www.rit.edu/cos/uphysics/uncertainties/Uncertaintiespart2.html

18. Exact numbers Adding/Subtracting: –Error does not change Multiplying: – Multiply the error by the same number

19. Significant Digits (Figures) How many significant digits are in: –12 –12.00 –12.001 –12000 –0.0001 –0.00012 –123456789 Only applies to measured values, not exact values (i.e. 2 oranges)

20. Significant Digits Cannot create precision: –1.0 * 2.0 = 2.0 –12 * 11 = 130 (not 131) –12.0 * 11 = 130 (still not 131) –12.0 * 11.0 = 131 Can keep digits for calculations, report with appropriate significant digits

21. Rounding If you have 2 significant digits: –1.11 -> ? –1.19 -> ? –1.14 -> ? –1.16 -> ? –1.15 -> ? –1.99 -> ? –1.155 -> ?

22. Quality Control/Assurance Calibrate “Instruments” Perform random checks on data Watch for “drift” Document all errors in Metadata!

23. Design of Sampling Random Stratified random Clustered Systematic Iterative

24. Number of Samples 30? Figure 2.7 from Environmental Monitoring and Characterization

25. Statistical Studies Is the sampling really random or uniform? –Bias –“Most data is collect near a road, a porta- poty, and a restaurant!” – Tom Stohlgren

26. Plots in RMNP

27. Plots in GSENM

28. Spatial Autocorrelation Used to determine type of sampling

29. Rounding If you have 2 significant digits: –1.11 -> 1.1 –1.19 -> 1.2 –1.14 -> 1.1 –1.16 -> 1.2 –1.15 -> 1.1 –1.99 -> 2.0 –1.155 -> 1.5