1. Lies, Damned Lies, and Statistics Hanford Performance Indicator Forum December 11, 2003 Steven S Prevette Fluor Hanford Safety and Health 509-373-9371 Steven_s_prevette@rl.gov http://www.hanford.gov/safety/vpp/trend.htm

2. Introduction Numbers abound in the Safety profession We have all seen great uses and misuses How can we make the best use of the data we have and make effective decisions that will improve the safety of ourselves, our coworkers, and our companies?

3. Liars Figure and Figures Lie 1 Here are the past 2 years of injury data. What is your interpretation? What is your prediction of future rates?

4. Liars Figure and Figures Lie 2 Let’s add a “trend line”. What is your interpretation now? What is your prediction of future rates?

5. Liars Figure and Figures Lie 3 Let’s add a 6 month moving average. What is your interpretation now? What is your prediction of future rates?

6. Interlude – Fun with Charts It appears I can make this data say anything I want it to say... Courses are taught on “chartsmanship” Can we interpret the data with a sound, rigorous method that is repeatable and gives credible results?

7. Liars Figure and Figures Lie 4 How about the Control Chart? - The data are stable, within certain bounds.

8. The Red Bead Experiment Many of you have experienced the “Red Bead Experiment” Results changed from person to person, but were random and predictable The data here were generated by the Red Beads Which chart gave the proper interpretation and predictions?

9. The Control Chart Separates random noise from signal Provides prediction capability When coupled with management theory and action can lead to significant improvements Does not require great costs to perform Actually has saved money

10. Management Theory For Improvement When there is a trend, find “special cause” and act upon the trend When there is no trend, find “common causes” and act upon the system, the overall performance

11. Pareto Charts Pareto charts can be used to analyze common causes – FH “Top 5 Occupation” initiative from 1998

12. What is a Trend? One point outside the control limits Two out of Three points two standard deviations above/below average Four out of Five points one standard deviation above/below average Seven points in a row all above/below average Ten out of Eleven points above/below average Seven points all increasing/decreasing. See http://www.hanford.gov/safety/vpp/trend.htm

13. Leading Indicators This system can be used with leading indicators and Behavior Based Safety A personal example – my group targeted computer ergonomics Goal was to improve behaviors on the computer, to reduce risk of Carpal Tunnel / Overuse Syndrome

14. Leading Indicator Example BEHAVIOR TO OBSERVEYESNO Backrest of chair is supporting person's back Forearms are parallel to floor Knees at angle of 90 - 110 degrees Legs supported by chair seat Person's eyes are 18 - 30 inches from screen Top of monitor at or below eye level Monitor viewed straight ahead with no neck turn Wrists are in proper position (Wrist Support in use, or wrists held at level they would be if there was a Wrist Support) Typing stand (document holder) at same height and distance as screen [leave blank if no stand in use] No glare of outside light, overhead lights, or work lights on screen.

15. Leading Indicator Results Initial rate was 10% safe behaviors Plotted on Control Chart Pareto chart of individual items established Safe behavior increased to 83% within 5 months, a significant increase

16. Barriers to Good Data Use Higher ups will use it as a “hammer” Subjected to quotas and targets imposed from above Fear (“accountability”) used as a “motivator” Actions and Explanations as a result of random fluctuations Perceived loss of control over portrayal of performance The Biggest Barrier = FEAR

17. Data Sources Worker and Customer Opinion Expert Review Process Measures –Top-Down (start with mission/vision) –Process Approach –Bottom-Up (start with available data)

18. Data Sources Process Product Service Schedule OUTCOMES Mission Progress Commitments Met Stewardship Profit Safety Satisfaction Compliance THEORY Cycle Time In-Process Inventory Procedure Waste Rework Idle Time Dollars Hours Materials Data Budget

19. More Fun with Numbers Boston Herald (Dec 23, 1996): "Murders Sink To 30-Year Low " "....major crime such as homicide is down. Over the last two years there has been a 52% reduction in shootings..... that's a dramatic difference....Police Commissioner Paul Evans has succeeded where others have fell short...."

20. More Fun with Numbers 2

21. “Murder rate rises in Boston” BOSTON (AP) — The number of murders in Boston rose in 2000 after years of steady decline, partially due to an alarmingly violent first six months. The final tally was 37 — up from 31 last year. [Police Commissioner] Evans noted that there were no homicides in October or November For 2001, AP reported a 67% increase

22. Conclusion – Data Sanity We can either react to numbers, with explanations of every percent change, with the inherent frustrations, fear, and failure Or We can understand our data, put it to good use, and apply valid management principles The choice is ours.