Defining Investigation Extent of Condition/Cause Scope a New Approach                                                      Steve Davis The Excellence Engine, LLC June 2010

a.k.a. How big is this problem? Extent of Condition/Cause a.k.a. How big is this problem?

Extent of Condition Transportability of condition - determine whether the same problem/condition exists elsewhere. Is the extent to which the condition may exist in other plant equipment, organizations, processes, or human performance. Objective .018

Why Perform an Extent of Condition Review? To identify down stream effects from the initial event. To identify possible event initiators and correct them prior to causing an additional event.

Extent of Cause Transportability of cause - determine whether the same root or underlying causes of the problem/condition may be affecting performance elsewhere (other plant processes, equipment, or human performance). Objective .018

Why Perform an Extent of Cause Review? To identify other places where this cause exists and correct them prior to causing an additional event.

The Problem Extent of Cause/Condition review too narrow. Extent of Cause/Condition review too broad. Extent of Cause/Condition review just right but is criticized as being too broad/narrow. Extent of Cause/Condition review assesses the wrong areas.

The Solution A Risk Based Approach to Setting Extent of Condition/Cause Assessment Scope

Risk Based Three Step Approach First, define other potential instances of the same/similar condition/cause(s). Second, determine the risk of a similar event occurring with the same/similar condition/cause(s) by assigning a probability of occurrence and level of consequence if no corrective or preventive actions are taken. Third, decide if action is required. Objective .018

Define Potential Same/Similar Conditions Restate the event / problem and critical setup conditions Define Same/Similar using the following criteria. –Same - Same Same Program [e.g., CAP] or Process [e.g., Surveillance Scheduling] or System [Auxiliary Steam] with the Same kind of problem/error/failure or defect. Same Object – Same Defect –Same - Similar Same Program [e.g., CAP vs. Self-Assessment] or Process [e.g., Surveillance Scheduling vs. 13-Week Scheduling] or System [Auxiliary Steam vs. Heater Drains] with a Similar kind of problem/error/failure or defect. Same Object – Similar Defect –Similar - Similar Similar Program [e.g., CAP vs. Self-Assessment vs. Management Oversight] or Process [e.g., Surveillance Scheduling vs. 13-Week Scheduling vs. WM] or System [Auxiliary Steam vs. Heater Drains vs. Main Steam] with a Similar kind of problem/error/failure or defect. Similar Object – Similar Defect Objective .018

Define Potential Same/Similar Causes Restate the problem statement and identified cause. (Typically just the Root Cause or principal Apparent Cause) Define Same/Similar using the following criteria. –Same - Same Same Same Same Cause – Same Impacts –Same - Similar Same Similar Same Cause – Similar Impact –Similar - Similar Similar Similar Similar Cause – Similar Impact Objective .018

Set the Scope of the Assessment Using the defined Same/Similar conditions/ cause(s): Determine the consequences for each defined potential condition/cause. Evaluate the chance of occurrence for each defined potential condition/cause. Identify the Risk for each defined potential condition/cause. Bound the Extent of Condition evaluation based on the defined risk. (e.g. No Risk = No evaluation)

Material Problems with the New Roof Nails in Tires Consequence Other Foreign Material Problems with the New Roof Roofing Nails In Driveway Roofing Nails In Other Places Same - Same Same - Similar Similar - Similar Objective .018 Roofer Foreign Material Mgt. Extent of cause Same - Same 360 Degree Approach to Extent of Condition/Cause

Suggested Extent of Condition Analysis Template Set up Conditions Problem / Event Extent of Condition Potential SAME – SAME Condition RISK MITIGATING ACTIONS Potential SAME – SIMILAR Condition POTENTIAL SIMILAR - SIMILAR CONDITION Objective .018 ©2008 The Excellence Engine

Suggested Extent of Cause Analysis Template Problem Statement Apparent / Root Cause Extent of Cause Potential SAME – SAME Impacts RISK MITIGATING ACTIONS Potential SAME – SIMILAR Impacts POTENTIAL SIMILAR - SIMILAR IMPACTS Objective .018 ©2008 The Excellence Engine

Extent of Condition/Cause Assessment Definition Matrix Objective .018 ©2008 The Excellence Engine

Extent of Condition/Cause Assessment Risk Matrix Objective .018 ©2008 The Excellence Engine

Extent of Condition/Cause Assessment Mitigating Actions Matrix Objective .018 ©2008 The Excellence Engine

Extent of Condition/Cause Assessment Risk to Action Matrix Objective .018 ©2008 The Excellence Engine

Example Problem A few gallons of hydrazine contaminated with sodium hydroxide waste was pumped from the secondary hydrazine addition tank to the Auxiliary Feedwater Storage Tank, which was being used to feed the Steam Generators while the plant was in Hot Standby. Objective .018

Extent of Condition Example Set up Conditions Problem / Event Extent of Condition An empty hydrazine drum filled with sodium hydroxide waste was not labeled per station requirements. The waste drum was later mistaken for a new drum of hydrazine and its contents were added to the secondary hydrazine addition tank. A few gallons of hydrazine contaminated with sodium hydroxide waste was pumped from the secondary hydrazine addition tank to the Auxiliary Feedwater Storage Tank, which was being used to feed the S/Gs while the plant was in Hot Standby. Potential SAME – SAME Condition RISK What other systems or tanks are contaminated due to transfer from the secondary hydrazine addition tank or other direct transfers from the contaminated waste drum? MITIGATING ACTIONS Potential SAME – SIMILAR Condition Are there any other mislabeled drums that could mistakenly be added to the same or other plant systems? POTENTIAL SIMILAR – SIMILAR CONDITION Are there any other chemical transfer tasks that could be mistakenly performed because of a labeling error? Objective .018 ©2008 The Excellence Engine

Extent of Condition/Cause Assessment Risk to Action Matrix Objective .018 ©2008 The Excellence Engine

Extent of Condition Example Set up Conditions Problem / Event Extent of Condition An empty hydrazine drum filled with sodium hydroxide waste was not labeled per station requirements. The waste drum was later mistaken for a new drum of hydrazine and its contents were added to the secondary hydrazine addition tank. A few gallons of hydrazine contaminated with sodium hydroxide waste was pumped from the secondary hydrazine addition tank to the Auxiliary Feedwater Storage Tank, which was being used to feed the S/Gs while the plant was in Hot Standby. Potential SAME – SAME Condition RISK What other systems or tanks are contaminated due to transfer from the secondary hydrazine addition tank or other direct transfers from the contaminated waste drum? Probable & Critical = HIGH RISK MITIGATING ACTIONS Evaluate & Correct Potential SAME – SIMILAR Condition Are there any other mislabeled drums that could mistakenly be added to the same or other plant systems? POTENTIAL SIMILAR – SIMILAR CONDITION Are there any other chemical transfer tasks that could be mistakenly performed because of a labeling error? Remote & Critical = MEDIUM RISK Mgmt. Discretion Objective .018 ©2008 The Excellence Engine

Extent of Cause Example Problem Statement Apparent / Root Cause Extent of Cause A few gallons of hydrazine contaminated with sodium hydroxide waste was pumped from the secondary hydrazine addition tank to the Auxiliary Feedwater Storage Tank, which was being used to feed the S/Gs while the plant was in Hot Standby. Willful violation (i.e. shortcut taken) on the part of the individual responsible for labeling the waste drum. Potential SAME – SAME Impacts RISK Has the same individual taken shortcuts when labeling other waste drums? MITIGATING ACTIONS Potential SAME – SIMILAR Impacts Has the same individual taken any other safety significant shortcuts? POTENTIAL SIMILAR – SIMILAR IMPACTS Are other individuals in the same work group willfully violating labeling requirements? Objective .018 ©2008 The Excellence Engine

Extent of Condition/Cause Assessment Risk to Action Matrix Objective .018 ©2008 The Excellence Engine

Potential SAME – SIMILAR Impacts POTENTIAL SIMILAR – SIMILAR IMPACTS Extent of Cause Example Problem Statement Apparent / Root Cause Extent of Cause A few gallons of hydrazine contaminated with sodium hydroxide waste was pumped from the secondary hydrazine addition tank to the Auxiliary Feedwater Storage Tank, which was being used to feed the S/Gs while the plant was in Hot Standby. Willful violation (i.e. shortcut taken) on the part of the individual responsible for labeling the waste drum. Potential SAME – SAME Impacts RISK Has the same individual taken shortcuts when labeling other waste drums? Probable & Critical = HIGH RISK MITIGATING ACTIONS Evaluate & Correct Potential SAME – SIMILAR Impacts Has the same individual taken any other safety significant shortcuts? Probable & Marginal = MEDIUM RISK POTENTIAL SIMILAR – SIMILAR IMPACTS Are other individuals in the same work group willfully violating labeling requirements? Improbable & Marginal = LOW No Action Objective .018 ©2008 The Excellence Engine

Risk Based Approach to Setting Extent of Condition/Cause Assessment Scope Value-Added Provides standardized approach for setting extent of condition/cause assessment scope. Makes thinking visible. Simplifies identifying appropriate corrective actions.

Questions??? Steve Davis Steve.davis@excellence-engine.com Objective .019 Problem: The Startup rate and Intermediate rate nuclear instruments were rendered unavailable when terminal voltage on batteries B and D unexpectedly fell below the minimum allowed voltage of 21 VDC during the performance of scheduled battery charger maintenance. The event resulted in an equipment damage near miss. Setup Conditions: During previous planned maintenance, the opposite train batteries were operated beyond their design basis limit. During the planned maintenance activity, monitoring of battery terminal voltage when the chargers were turned off was only performed during the day shift, not during the night shift. Key individuals had a knowledge deficiency as to how battery voltage would decay An identified battery monitoring deficiency with the battery operating procedure was not corrected Same – Same Scope: Batteries A, B, C, D, Other compensatory measures with Battery A, B, C, D planned maintenance. Battery A, B, C, D operating procedure. Questions Where else do we have the potential for an unexpected battery condition (operating beyond design, low/high voltage, etc,) during battery charger maintenance on Batteries A, B, C, and D due to inadequate compensatory measures? Are there other monitoring deficiencies in battery A, B, C, D operating procedure? Same – Similar Other Station Batteries, both safety related and non-safety related. Licensed Operators, Electrical Maintenance, Planners, System Engineer. Other battery operating procedures Questions: Where else do we have the potential for an unexpected battery condition (operating beyond design, low/high voltage, etc,) during battery charger maintenance on other station batteries due to inadequate compensatory measures? Are there other deficiencies with the battery operating procedure? What other battery knowledge deficiencies exist with key station personnel? Similar – Similar All planned maintenance packages that require compensatory measures Other station personnel that have oversight for battery operation or maintenance Other procedures that contain battery operating instructions What other planned maintenance activities have the potential result in an unexpected condition because of inadequate compensatory measures? What other station personnel have battery knowledge deficiency. What other station operating procedures have that contain battery instructions have deficiencies? Problem Root Cause(s) High work control performance standards and practices have not been enforced by ensuring procedures and work practices meet the station’s needs, and are used. Work control process (planning, package development, scheduling, reviews, approvals, etc.). None, the root cause statement covers the entire work control process. Other work processes that may affect plant equipment. Question: Is the station enforcing high performance standards in the projects and modification process? Projects and modifications both affect installed plant equipment. Other station program/processes Where at the station is the management team not enforcing high program/process performance standards (e.g., Training, Safety, Radiation Protection)? Steve Davis Steve.davis@excellence-engine.com 315-593-3447 315-532-4742 Chet Rowe Chet.rowe@excellence-engine.com 315 207-3018 315-591-2159