1. Aileen Sullivan Director, Performance Improvement, OPG October 2017
Successful use of Nuclear Safety Culture Assessment to improve Nuclear Refurbishment Project
Aileen Sullivan
Director, Performance Improvement, OPG
October 2017

2. Darlington Refurbishment
The Darlington Refurbishment Program (DRP) is a multi-year, multi-phase mega-program to enable safe and reliable operation of Darlington until ~2055
Includes the replacement or rehabilitation of life-limiting components as well as upgrades to meet regulatory requirements
Refurbishment of the four units will span 112 months, to February 2026, including 40 months for Unit 2 from October 2016 to February 2020
OPG has a high level of confidence in the DRP cost estimate of $12.8B, which includes contingency, capitalized interest and escalation
According to the Conference Board of Canada, the construction phase alone is expected to generate $14.9 billion in economic benefits to Ontario. At its peak, refurbishment will create 11,700 jobs per year, with an average of 8,800 annually between 2014 and It is also expected to increase household revenues in Ontario by $8.5 billion.

3. DRP Economic Impact

4. Why did we decide to assess Nuclear Safety Culture
Not a regulatory, project or company requirement
Important to our CEO and Project Executive Team to:
determine the safety culture established before project execution began to take proactive steps to improve attributes of the culture that need addressing
Identify improvements needed to onboarding and nuclear safety training for the incoming large influx of a number of new to nuclear transient workers expected from the union halls in the first 6 months
set a baseline to measure safety culture 1 year into execution – after workforce has increased by more than 50% - to detect any decline and to continuously strengthen the health of the culture
Provide opportunity to listen to the workforce – internal and vendors – assess health of working relationships
OPEX has shown that poor safety culture could negatively impact such a project
Senior leadership value the process and its insights to improve culture

5. NSC Assessment Method
Used same assessment method that was used to assess NPP
Used INPO NSC survey based on 10 traits – validated survey with reference data
Modified workgroups to include construction work groups/titles
4 week survey + interviews + focus groups + document/data review
Paper and electronic survey due to large vendor workforce who did not have access to computers
Diverse team make-up represented workforce
Analysis done by organizational psychologist
Interviews and focus group - >50% front line workers; representative sample of leaders and workers from the 9 companies/groups
Reviewed 81 question survey (1,997 respondents), > 750 comments, > 90 interviews, 8 field observations and ~ 1500 data points
Prepared set of questions for every trait and attribute – insights scored:
Positive Observation if the behaviour or employee’s belief exceeds the expected industry standard
Neutral Observation if the behaviour or employee’s belief meets the industry standard
Negative Observation if the behaviour or employee’s belief is perceived by the assessor as not meeting industry standard
Why did you decide to use the method you chose? How did you have to change? What training did you have to carry out?
Used same assessment method that was used to assess NPP – also had reference group data for other vendor companies (INPO)
Leaders already trained the process – used since 2003 – follow procedure
Used INPO NSC survey based on 10 traits
Modified workgroups to include construction work groups/titles
4 week survey + interviews + focus groups + document/data review
Paper and electronic survey due to large vendor workforce who did not have access to computers – scheduled time for vendors to complete survey
Senior VP – Fleet O&M as team leader, organizational psychologist – assistant team leader, internal and external members of team, accelerate candidates (development), new and experienced team members representing vertical slice of workforce including 1 person from each of 9 companies/groups represented (ownership of and input to the results)
Analysis done by organizational psychologist – insights brought to assessment team during training day to help inform the interview and focus group part of the assessment
Interviews and focus group - >50% front line workers; representative sample of leaders and workers from the 9 companies/groups
Prepared set of questions for every trait and attribute – insights gleaned were scored – strength, meets standard, AFI
4 week survey, 2 weeks analysis, 1 week assessment, report out draft to Refurb team on Friday, communication to staff the following week and Nuclear Executive team a few weeks later
Entered into self assessment system and action tracking system

6. NSC Assessment Method continued
Presentation Title
NSC Assessment Method continued
For the purposes of the NSCA methodology the following terms are used exclusively in characterizing the issues and themes:
Strength – A consistently demonstrated belief, action, or process; related to the organizations values or behavior that exceeds the industry norm as established in the related INPO traits and attributes.
Positive Observation – Narrowly focused behaviours or attributes that the station is doing well, and may have the opportunity to leverage on a broader scale to improve safety culture or station performance.
Weakness – A consistently demonstrated belief, action, or process; related to the organizations values or behavior, that does not meet the industry norm as established in the related INPO traits and attributes and requires aggressive management attention and correction. In most station self-assessment programs, a Weakness is the equivalent to an Area For Improvement.
Negative Observation – A narrowly focused issue that is confined to a small sample of the population (that is a specific work group or level in the organization) and/or may represent a potential future performance deficiency which requires management attention.
Preliminary Debrief to Refurb lead team on Friday
Communication to staff the following week
Draft report within 30 days and final report completed within 60 days to the Nuclear Executive team
Entered into self assessment system and Refurb action tracking system
Communicated results face to face with regulator
Got feedback from team members and completed lessons learned self assessment
Finding Definitions:
For the purposes of the NSCA methodology the following terms are used exclusively in characterizing the issues and themes:
Strength – A consistently demonstrated belief, action, or process; related to the organizations values or behavior that exceeds the industry norm as established in the related INPO traits and attributes.
Positive Observation – Narrowly focused behaviours or attributes that the station is doing well, and may have the opportunity to leverage on a broader scale to improve safety culture or station performance.
Weakness – A consistently demonstrated belief, action, or process; related to the organizations values or behavior, that does not meet the industry norm as established in the related INPO traits and attributes and requires aggressive management attention and correction. In most station self-assessment programs, a Weakness is the equivalent to an Area For Improvement.
Negative Observation – A narrowly focused issue that is confined to a small sample of the population (that is a specific work group or level in the organization) and/or may represent a potential future performance deficiency which requires management attention.

7. Success Factors and Lessons Learned
1. Communication and Engagement success
93% response rate on NSC Survey – 9 companies/groups involved in the project, 1,997 respondents – HOW?
Communication plan – SPOC for each of 9 companies/groups
Weekly planning team meetings
Met with unions to solicit their support
Face to face and written communication – why it is important to the company and the project
Survey responses went to INPO – anonymous survey – confidential
Written and electronic responses
Scheduled time for contractors to complete written surveys – sealed in envelope and went to an independent data entry group - confidentiality agreement – input data to INPO database
Regular tracking of % complete with further communications and encouragement
2. Team make-up, training, mentoring, data entry support

8. 3. Developing SMART action plans
Selected a second team of key stakeholders – those who would own the actions through implementation and representation from the assessment team – Three 1/2 day facilitated workshop
Divided into 5 teams – 1 per finding
Brainstormed potential actions – prioritized top 3
Presented back to larger group – challenge panel
Agreed top action themes to go after
Groups developed SMART actions – measurable with who, by when
Presented back to larger group + team leader/host sponsor
Finalized plan and presented to Refurb Executive Team
Results and action plan presented to Nuclear Executive Team
Actions entered into Refurb Mngt Oversight database for tracking

9. 4. Implemented some quick wins
Show the organization that they were heard (entry system, working conditions, parking, blue card (employee concerns/suggestions)
Established Nuclear Projects Nuclear Safety Culture Monitoring Panel
Produced a handbook on NSC traits and rolled out to all Refurb staff/Trades.
Conducted Leadership Alignment workshops to build on the Nuclear Safety Culture Assessment focusing on team work, behaviours and communications between all parties.
Adopted daily nuclear safety culture message (before all meetings and start of shift) – replaced construction principles with safety attributes
Added nuclear safety culture module to onboarding training
Produced video from CNO and added to Nuclear General Training (CBT)
5. Follow-up survey and focus groups planned for Q along with operating station