Writing Findings, Observations, and the Executive Summary Susan Aderholdt, TSDF Lead and Corrective Actions Coordinator, DOECAP Operations Team Analytical Services Program Workshop September 19-22, 2016

Deficiency Description Paragraph Parts of a Finding Finding Statement Finding ID Priority Level M4-16####-A: ABC does not control all spreadsheet revisions. (Priority II) (U.S. Department of Defense/DOE Consolidated Quality Systems Manual (QSM) for Environmental Laboratories, Version 5.0, Module 2, Section #.#.#.#; Procedure ABC‑123, Rev. 4, Section 2.0) ABC shall control spreadsheet revisions.   The auditor’s review of ABC’s spreadsheets identified two types of spreadsheets that are used during sample analysis which are not maintained under revision control. Procedure ABC-123, Documents, provides controls for developing and revising documents, including spreadsheets. Requirement Citations Requirement Statement Deficiency Description Paragraph Deficiency Statement (first sentence)

Look Up the Requirement If you identified an issue that might be a finding, look up the actual requirement in the source document before you start writing your finding against it. DOECAP audit checklists are not requirements. Make sure you understand the requirement, especially if it’s from a site procedure. Many procedures have applicability statements that might negate the requirement for specific circumstances.

What is wrong with this finding? WO-16XXXX-A: Aisle space is not adequately maintained in a less-than-90-day mixed waste storage area. (Priority II) No reference

What is wrong with this finding? WO-16XXXX-A: Aisle space is not adequately maintained in a less-than-90-day mixed waste storage area. (Priority II) (XXX Department of Environment and Conservation Rule, Sections 0400-12-01-.03(4)(e)(2)(iv) and 0400-12-01-.05(3)(f)) Need to add reference

What is wrong with this finding? M3-16XXXX-C: Procedure XYZ-123 requirements are not being met. In addition, this procedure has not been updated to reflect changes based on the new LIMS functions. (Priority II) (Procedure XYZ-123, Rev. 02, Receipt, Use, and Management of Chemicals and Reagents, Section 2) A finding is one sentence.

What is wrong with this finding? M3-16XXXX-C: Procedure XYZ-123, Receipt, Use, and Management of Chemicals and Reagents, has not been updated to reflect changes based on new LIMS functions. (Priority II) (Procedure XYZ-123, Rev. 02, “Receipt, Use, and Management of Chemicals and Reagents,” Section 2) Edited to remove the extra information. That information can go into the discussion below.

What is wrong with this finding? WO-16XXXX-A: Aisle space is not adequately maintained in a less-than-90-day mixed waste storage area. (Priority II) (XXX Department of Environment and Conservation Rule, Sections 0400-12-01-.03(4)(e)(2)(iv) and 0400-12-01-.05(3)(f)) A mixed waste storage area in the Incineration Building has eleven 55-gallon containers of hazardous waste incinerator ash. The containers are placed three deep, with no aisle space provided, resulting in obstruction of movement of personnel in the event of an emergency. No requirements statement

What is wrong with this finding? WO-16XXXX-A: Aisle space is not adequately maintained in a less- than-90-day mixed waste storage area. (Priority II) (XXX Department of Environment and Conservation Rule, Sections 0400-12-01- .03(4)(e)(2)(iv) and 0400-12-01-.05(3)(f)) Sufficient aisle space must be maintained to allow the unobstructed movement of personnel, fire protection equipment, spill control equipment, and decontamination equipment to any area of facility operations in the event an emergency. A mixed waste storage area in the Incineration Building has eleven 55- gallon containers of hazardous waste incinerator ash. The containers are placed three deep, with no aisle space provided, resulting in obstruction of movement of personnel in the event of an emergency. Must have a requirements statement.

What is wrong with this finding? SH-16XXXX-B: Several issues were noted regarding proper fire extinguisher control. A fire extinguisher intended for fire watch use was identified that was not clearly marked “Fire Watch” as required by ABC procedures. Additionally, a mounted fire extinguisher was identified in the same area which did not fully meet NFPA requirements for a mounted fire extinguisher (Priority II). (Procedure XX-YY-ZZ-100, ABC Health and Safety Program, section 4.12.12.5; and National Fire Protection Association (NFPA) 10 Chapter 5) A finding is a single sentence

What is wrong with this finding? SH-16XXXX-B: Several issues were noted regarding proper fire extinguisher control. (Priority II) (Procedure XX-YY-ZZ-100, ABC Health and Safety Program, section 4.12.12.5; and National Fire Protection Association [NFPA] 10 Chapter 5) Rewrote to make one sentence and moved everything else into the discussion

Talking About Findings and Observations in the Executive Summary In the summary paragraph for each review area Closed Previous Findings: Provide one sentence that states how many previous findings are closed. You don’t need to say anything else about closed findings. Previous Findings That Remain Open: Provide one sentence that identifies the finding and include the finding identifier in brackets at the end of the sentence. New Findings: Provide a one-sentence statement or phrase about each new finding and end it with the finding identifier in brackets. If there are Priority I and Priority II findings in the report note if it is a Priority I or II. (This is new) Observations: Provide one sentence that identifies the number of observations for that review area. Don’t provide the observations or the identifiers.

Executive Summary – Key Findings Rollup Of concern is the failure of the laboratory to thoroughly review and implement contract modifications. Of other concern are the findings in data quality for organics and Inorganics that could result in non-compliant data. Several key findings were noted. One was due to polychlorinated biphenyl (PCB) waste being stored for more than one year, another involved improper storage of hazardous waste, and a third was due to failure to properly segregate packing material from waste material. These issues, as well as a few others, are noted in the body of this report. Although we have a place designated in the draft report that you receive in the pre-audit package, Most auditors, with a few exceptions are not rolling up and noting key findings in the executive summary. The lead auditor with help from the audit team must look at the findings and note in the appropriate place what the key findings are and make a statement about them. They should be the findings of the most concern. If you have several very minor findings you need to make a statement such as there were no findings of significance to be noted as key findings. I have included two examples here, one a lab and one a TSDF.

Executive Summary – What is Wrong? The ABC radioactive materials license (RML) authorizes sufficient capacity to support DOE’s analytical needs. ABC receives nonradiological samples from WRPS that have been free-released from the site, as well as samples containing technetium-99. ABC uses waste brokers to manage its outgoing shipments and final disposal of laboratory waste. The audit team closed four of seven previous findings with regard to ABC’s failure to meet the radiation safety officer (RSO) and backup RSO training requirements [M6-15XXXX-C], incorrect and missing names on the RML [M6-15XXXX-E], failure to meet the laboratory’s inspection schedules [M6-15XXXX-F], and failure to review the Chemical Hygiene Plan within the required timeframe [M6-15XXXX-G]. Do not put details of closed findings in the executive summary

Executive Summary – What is Wrong? The ABC radioactive materials license (RML) authorizes sufficient capacity to support DOE’s analytical needs. ABC receives nonradiological samples from WRPS that have been free-released from the site as well as samples containing technetium-99. ABC uses waste brokers to manage its outgoing shipments and final disposal of laboratory waste. The audit team closed four of seven previous findings. with regard to ABC’s failure to meet the radiation safety officer (RSO) and backup RSO training requirements [M6-15XXXX-C], incorrect and missing names on the RML [M6-15XXXX-E], failure to meet the laboratory’s inspection schedules [M6-15XXXX-F], and failure to review the Chemical Hygiene Plan within the required timeframe [M6-15XXXX-G]. Sentence needs to end with the audit team closed four of seven previous findings then have a sentence for each finding left open with the number in brackets at the end. We do not talk about the closed findings in the executive summary. Also, there is no mention of how many new findings there are, there is no finding statement with the number in brackets. No mention of observations either.

Executive Summary – What is Wrong? ABC waste operations, receipt, traceability, and regulatory compliance continue to be effective. The auditors determined that personnel are competent and knowledgeable in carrying out their tasks. The auditors reviewed waste disposal and storage operations. There were no previous findings. One new finding was issued regarding satellite accumulation area container control inside the radiological restricted area [WO-16XXXX-A]; however, ABC conducted immediate corrective actions. Three observations were identified: OWO-16XXXX-A concerning satellite accumulation area container secondary containment, OWO-16XXXX-B about lack of hazardous waste warning signs around the mixed waste facility, and OWO-16XXXX-C regarding observed mixed waste with the potential for weather dispersal. Do not put in details about observations

Executive Summary – What is Wrong? ABC waste operations, receipt, traceability, and regulatory compliance continue to be effective. The auditors determined that personnel are competent and knowledgeable in carrying out their tasks. The auditors reviewed waste disposal and storage operations. There were no previous findings. One new finding was issued regarding satellite accumulation area container control inside the radiological restricted area [WO-16XXXX-A]. however, ABC conducted immediate corrective actions. Three observations were identified: OWO-16XXXX-A concerning satellite accumulation area container secondary containment, OWO-16XXXX-B about lack of hazardous waste warning signs around the mixed waste facility, and OWO-16XXXX-C regarding observed mixed waste with the potential for weather dispersal. However, ABC conducted immediate corrective actions does not need to be in executive summary. Removed details of observations. All you need to say is Three observations were identified.

Executive Summary – What is Wrong?  ABC continues to actively pursue safety excellence in all operations and uses internal and external self-assessments to improve their performance. ABC management supports a questioning attitude of workers and listens to workers on safety-related issues. The DOECAP auditors attended the monthly ABC Safety Committee meeting where past and present safety-related issues were discussed, status on progress being made or completion, and how to continue to mature their health and safety program. Also, the DOECAP auditors observed an emergency training drill involving coordination with external flight-for-life services. The emergency preparedness drill was commended by the DOECAP as a proactive approach for worker safety. No mention of findings previous or new.

Executive Summary – What is Wrong?  ABC continues to actively pursue safety excellence in all operations and uses internal and external self-assessments to improve their performance. ABC management supports a questioning attitude of workers and listens to workers on safety related issues. The DOECAP auditors attended the monthly ABC Safety Committee meeting, where past and present safety- related issues were discussed, status on progress being made or completion, and how to continue to mature their health and safety program. Also, the DOECAP auditors observed an emergency training drill involving coordination with external flight-for-life services. The emergency preparedness drill was commended by the DOECAP as a proactive approach for worker safety. ABC supports the local community in providing ambulatory and emergency medical technicians as first responder support for remote accident response on Highway ## and has been recognized by the local community. One previous finding was closed. Five observations were identified, and two new findings were issued: Lack of formal approval of job hazard analyses [SH-16XXXX-A]. Incorrect size fire extinguisher mounted in the XYZ area [SH-16XXXX-B]. Edited to add findings and observations comment.

Executive Summary-What is Wrong? The auditor observed that ABC performs isotopic and total metals analysis by inductively coupled plasma (ICP)-mass spectrometry, metals analysis by ICP-atomic emission spectroscopy, and mercury by cold vapor atomic absorption. ABC is also capable of performing anions analysis by ion chromatography, conductivity, pH, and fluoride by ion specific electrode. The ABC Metals Department personnel are trained and knowledgeable with regard to the preparation and analysis of samples using the established protocols. One previous Priority II finding was closed. The following two Priority I and one Priority II findings were issued: • ABC failed to receive acceptable results for two consecutive rounds of MAPEP PT for the determination of selenium in a soil matrix (Priority I) [M3-16XXXX-A]. • ABC failed to receive acceptable results for two consecutive rounds of MAPEP PT for the determination of arsenic in a soil matrix (Priority I) [M3-16XXXX-B]. • ABC’s procedures do not accurately reflect the current activities (Priority II) [M3-16XXXX-C]. Nothing is wrong.

Avoid Any Variation of “Should” “Should” comes in many forms. Avoid all of them when you’re writing an observation: Should Is needed Might wish to consider Preferable Recommending a best management practice (BMP) without citing the source These words and phrases imply that DOE is telling the facility what to do, which is not appropriate in a DOECAP report. However, you can use these words IF they are part of a quote.

What is wrong with this observation? An interference study was performed for NIOSH 7303 by ICP/MS; however, a comparable study for ICP/AES has not been performed. Prior to analyzing industrial hygiene samples via ICP/AES, this study should be documented. Do not tell the facility what they should do

What is wrong with this observation? An interference study was performed for NIOSH 7303 by ICP/MS; however, a comparable study for ICP/AES has not been performed. Prior to analyzing industrial hygiene samples via ICP/AES, this study should be documented. Edited to remove the should statement.

What is wrong with this observation? The laboratory is recommended to include instructions for preparing incremental sampling method related to advance preparation of sub-sampling mercury soil samples.

What is wrong with this observation? The laboratory does not include instructions for preparing an incremental sampling method related to advance preparation for sub-sampling of mercury soil samples. Without those instructions, the advance preparation might not be performed correctly by new laboratory personnel. Rewrote to make a clear statement without using the word recommended. This is just a statement to say the lab is not doing something. It is an observation,

Questions?